|
65077
|
2280
|
5
|
2026-05-20T15:09:38.441706+00:00
|
/Users/lukas/.screenpipe/data/data/2026-05-20/1779 /Users/lukas/.screenpipe/data/data/2026-05-20/1779289778441_m2.jpg...
|
Firefox
|
Inbox - kovaliklukas@gmail.com - Gmail — Personal
|
1
|
mail.google.com/mail/u/0/#inbox
|
monitor_2
|
NULL
|
NULL
|
NULL
|
NULL
|
Inbox - [EMAIL] - Gmail
Inbox - [EMAIL] - Gmail
Cl Inbox - [EMAIL] - Gmail
Inbox - [EMAIL] - Gmail
Close tab
DXP4800PLUS-B5F8
DXP4800PLUS-B5F8
(191) TED LASSO BEST SCENES - PART 2. The Most Heartwarming Ted Lasso Show Moments. - YouTube
(191) TED LASSO BEST SCENES - PART 2. The Most Heartwarming Ted Lasso Show Moments. - YouTube
LinkedIn Login, Sign in | LinkedIn
LinkedIn Login, Sign in | LinkedIn
Ambulatory sheet - Paperless-ngx
Ambulatory sheet - Paperless-ngx
New Tab
New Tab
💬1 - Apple MacBook Pro 14" Space Black M5 Max | Laptop.bg - Технологията с теб
💬1 - Apple MacBook Pro 14" Space Black M5 Max | Laptop.bg - Технологията с теб
Apple Mac Studio M4 Max | Desktop.bg - Мощни PC Гейминг конфигурации, Персонални компютри на изплащане за игри и работа
Apple Mac Studio M4 Max | Desktop.bg - Мощни PC Гейминг конфигурации, Персонални компютри на изплащане за игри и работа
Welcome to authentik! - authentik
Welcome to authentik! - authentik
Завеждане на щета онлайн | Euroins
Завеждане на щета онлайн | Euroins
New Tab
Customize sidebar
Open Google Gemini (⌃X)
Open history (⇧⌘H)
Open bookmarks (⌘B)
Bitwarden
None selected
Skip to content
Skip to content
Using Gmail with screen readers
Using Gmail with screen readers
Main menu
Gmail
Search
Search
Search mail
Advanced search options
Search mail
Support
Settings
Ask Gemini
Google apps
Google Account: Lukáš Koválik ([EMAIL])
Compose
Labels
Labels
Inbox
Inbox
Starred
Starred
Snoozed
Snoozed
Important
Important
Sent
Sent
Drafts 8 unread
Drafts
8
Purchases has menu
Purchases
Social 5224 unread has menu
Social
5,224
Updates 8834 unread has menu
Updates
8,834
Forums 6161 unread has menu
Forums
6,161
Promotions 38854 unread has menu
Promotions
38,854
More labels
More
Labels
Labels
Create new label
Labels
Labels
[Imap]/Nevyžiadaná pošta has menu
[Imap]/Nevyžiadaná pošta
arch has menu
arch
Deleted Items has menu
Deleted Items
Fibank 1229 unread has menu
Fibank
1,229
FL 6 unread has menu
FL
6
Hardware & Software has menu
Hardware & Software
HOSTING 5 unread has menu
HOSTING
5
Infected Items has menu
Infected Items
jiminny-github 7560 unread has menu
jiminny-github
7,560
Junk E-mail 219 unread has menu
Junk E-mail
219
Kontakty has menu
Kontakty
Sent Items has menu
Sent Items
WORK 848 unread has menu
WORK
848
z centra 1274 unread has menu
z centra
1,274
More labels
More
Select
Refresh
More email options
Show more messages
1
–
50
of
21,333
Newer
Older
Toggle split pane mode
Input tools on/off (Ctrl-Shift-K)
Select input tool
Conversations
Conversations
Primary
Primary
Primary
Promotions, 50 new messages,
Promotions, 50 new messages,
50 new
Promotions
Social, 18 new messages,
Social, 18 new messages,
18 new
Social
Updates
Updates
Updates
CircleCI Builds, [CircleCI] Workflow canceled: jiminny / app on JY-20676-delete-report-related-objects, 17:06, app / build_accept_deploy jiminny / app 2026-05-20T13:56:47Z JY-20676-delete-report-related-objects #58634 Commit push by Lukas Kovalik (LakyLak) 4f15440 JY-20676 fix tests Authored by Lukas Kovalik (.
Not starred
Important mainly because it was sent directly to you.
CircleCI Builds
[CircleCI] Workflow canceled: jiminny / app on JY-20676-delete-report-related-objects - app / build_accept_deploy jiminny / app 2026-05-20T13:56:47Z JY-20676-delete-report-related-objects #58634 Commit push by Lukas Kovalik (LakyLak) 4f15440 JY-20676 fix tests Authored by Lukas Kovalik (
[CircleCI] Workflow canceled: jiminny / app on JY-20676-delete-report-related-objects
-
app / build_accept_deploy jiminny / app 2026-05-20T13:56:47Z JY-20676-delete-report-related-objects #58634 Commit push by Lukas Kovalik (LakyLak) 4f15440 JY-20676 fix tests Authored by Lukas Kovalik (
17:06
CircleCI Builds, [CircleCI] Workflow failed: jiminny / app on JY-20676-delete-report-related-objects, 15:18, app / build_accept_deploy jiminny / app 2026-05-20T12:03:29Z JY-20676-delete-report-related-objects #58619 Commit push by Lukas Kovalik (LakyLak) 84a3b9c JY-20676 code review suggestions Authored by.
Not starred
Important mainly because it was sent directly to you.
CircleCI Builds
[CircleCI] Workflow failed: jiminny / app on JY-20676-delete-report-related-objects - app / build_accept_deploy jiminny / app 2026-05-20T12:03:29Z JY-20676-delete-report-related-objects #58619 Commit push by Lukas Kovalik (LakyLak) 84a3b9c JY-20676 code review suggestions Authored by
[CircleCI] Workflow failed: jiminny / app on JY-20676-delete-report-related-objects
-
app / build_accept_deploy jiminny / app 2026-05-20T12:03:29Z JY-20676-delete-report-related-objects #58619 Commit push by Lukas Kovalik (LakyLak) 84a3b9c JY-20676 code review suggestions Authored by
15:18
CircleCI Builds, [CircleCI] Workflow canceled: jiminny / app on JY-20613-allow-owner-role-on-team-setup, 14:44, app / build_accept_deploy jiminny / app 2026-05-20T11:44:29Z JY-20613-allow-owner-role-on-team-setup #58611 Commit push by Lukas Kovalik (LakyLak) 3642c93 JY-20613 remove recoder and voice role from.
Not starred
Important mainly because it was sent directly to you....
|
[{"role":"AXRadioButton","text [{"role":"AXRadioButton","text":"Inbox - kovaliklukas@gmail.com - Gmail","depth":4,"bounds":{"left":0.0,"top":0.0518755,"width":0.06881649,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"Inbox - kovaliklukas@gmail.com - Gmail","depth":5,"bounds":{"left":0.013297873,"top":0.06304868,"width":0.06898271,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Close tab","depth":5,"bounds":{"left":0.05651596,"top":0.05905826,"width":0.007978723,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXRadioButton","text":"DXP4800PLUS-B5F8","depth":4,"bounds":{"left":0.0,"top":0.08459697,"width":0.06881649,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"DXP4800PLUS-B5F8","depth":5,"bounds":{"left":0.013297873,"top":0.09577015,"width":0.036901597,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"(191) TED LASSO BEST SCENES - PART 2. The Most Heartwarming Ted Lasso Show Moments. - YouTube","depth":4,"bounds":{"left":0.0,"top":0.11731844,"width":0.06881649,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"(191) TED LASSO BEST SCENES - PART 2. The Most Heartwarming Ted Lasso Show Moments. - YouTube","depth":5,"bounds":{"left":0.013297873,"top":0.12849163,"width":0.18218085,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"LinkedIn Login, Sign in | LinkedIn","depth":4,"bounds":{"left":0.0,"top":0.15003991,"width":0.06881649,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LinkedIn Login, Sign in | LinkedIn","depth":5,"bounds":{"left":0.013297873,"top":0.16121309,"width":0.05668218,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Ambulatory sheet - Paperless-ngx","depth":4,"bounds":{"left":0.0,"top":0.18276137,"width":0.06881649,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Ambulatory sheet - Paperless-ngx","depth":5,"bounds":{"left":0.013297873,"top":0.19393456,"width":0.059507977,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"New Tab","depth":4,"bounds":{"left":0.0,"top":0.21548285,"width":0.06881649,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"New Tab","depth":5,"bounds":{"left":0.013297873,"top":0.22665602,"width":0.014960106,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"💬1 - Apple MacBook Pro 14\" Space Black M5 Max | Laptop.bg - Технологията с теб","depth":4,"bounds":{"left":0.0,"top":0.2482043,"width":0.06881649,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"💬1 - Apple MacBook Pro 14\" Space Black M5 Max | Laptop.bg - Технологията с теб","depth":5,"bounds":{"left":0.013297873,"top":0.25937748,"width":0.14727394,"height":0.011572227},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Apple Mac Studio M4 Max | Desktop.bg - Мощни PC Гейминг конфигурации, Персонални компютри на изплащане за игри и работа","depth":4,"bounds":{"left":0.0,"top":0.28092578,"width":0.06881649,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Apple Mac Studio M4 Max | Desktop.bg - Мощни PC Гейминг конфигурации, Персонални компютри на изплащане за игри и работа","depth":5,"bounds":{"left":0.013297873,"top":0.29209897,"width":0.2365359,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Welcome to authentik! - authentik","depth":4,"bounds":{"left":0.0,"top":0.31364724,"width":0.06881649,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Welcome to authentik! - authentik","depth":5,"bounds":{"left":0.013297873,"top":0.32482043,"width":0.05900931,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Завеждане на щета онлайн | Euroins","depth":4,"bounds":{"left":0.0,"top":0.3463687,"width":0.06881649,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Завеждане на щета онлайн | Euroins","depth":5,"bounds":{"left":0.013297873,"top":0.3575419,"width":0.0653258,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Tab","depth":4,"bounds":{"left":0.0028257978,"top":0.38068634,"width":0.06333112,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Customize sidebar","depth":6,"bounds":{"left":0.0028257978,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open Google Gemini (⌃X)","depth":6,"bounds":{"left":0.013796543,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open history (⇧⌘H)","depth":6,"bounds":{"left":0.024933511,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open bookmarks (⌘B)","depth":6,"bounds":{"left":0.036070477,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bitwarden","depth":6,"bounds":{"left":0.04720745,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"None selected","depth":8,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Skip to content","depth":10,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Skip to content","depth":11,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Using Gmail with screen readers","depth":10,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXStaticText","text":"Using Gmail with screen readers","depth":11,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Main menu","depth":11,"bounds":{"left":0.07280585,"top":0.058260176,"width":0.015957447,"height":0.03830806},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":true},{"role":"AXLink","text":"Gmail","depth":12,"bounds":{"left":0.09009308,"top":0.061452515,"width":0.036236703,"height":0.035115723},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Search","depth":12,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Search","depth":13,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXTextField","text":"Search mail","depth":18,"bounds":{"left":0.17220744,"top":0.06943336,"width":0.18916224,"height":0.016360734},"on_screen":true,"help_text":"","role_description":"text field","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Advanced search options","depth":12,"bounds":{"left":0.37466756,"top":0.058260176,"width":0.01861702,"height":0.03671189},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Search mail","depth":12,"bounds":{"left":0.15392287,"top":0.058260176,"width":0.01861702,"height":0.03671189},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":false,"is_focused":false,"is_selected":false},{"role":"AXMenuButton","text":"Support","depth":12,"bounds":{"left":0.66356385,"top":0.061452515,"width":0.013297873,"height":0.031923383},"on_screen":true,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXMenuButton","text":"Settings","depth":12,"bounds":{"left":0.6781915,"top":0.061452515,"width":0.013297873,"height":0.031923383},"on_screen":true,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Ask Gemini","depth":12,"bounds":{"left":0.6921542,"top":0.061452515,"width":0.013297873,"height":0.031923383},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Google apps","depth":14,"bounds":{"left":0.70611703,"top":0.061452515,"width":0.013297873,"height":0.031923383},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Google Account: Lukáš Koválik (kovaliklukas@gmail.com)","depth":14,"bounds":{"left":0.72207445,"top":0.061452515,"width":0.013297873,"height":0.031923383},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Compose","depth":9,"bounds":{"left":0.071476065,"top":0.10933759,"width":0.04737367,"height":0.031923383},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Labels","depth":13,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Labels","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Inbox","depth":17,"bounds":{"left":0.09009308,"top":0.15003991,"width":0.012466756,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Inbox","depth":18,"bounds":{"left":0.09009308,"top":0.15003991,"width":0.012466756,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Starred","depth":17,"bounds":{"left":0.09009308,"top":0.16919394,"width":0.015625,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Starred","depth":18,"bounds":{"left":0.09009308,"top":0.16919394,"width":0.015625,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Snoozed","depth":17,"bounds":{"left":0.09009308,"top":0.18834797,"width":0.018284574,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Snoozed","depth":18,"bounds":{"left":0.09009308,"top":0.18834797,"width":0.018284574,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Important","depth":18,"bounds":{"left":0.09009308,"top":0.207502,"width":0.020777926,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Important","depth":19,"bounds":{"left":0.09009308,"top":0.207502,"width":0.020777926,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Sent","depth":17,"bounds":{"left":0.09009308,"top":0.22665602,"width":0.009640957,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Sent","depth":18,"bounds":{"left":0.09009308,"top":0.22665602,"width":0.009640957,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Drafts 8 unread","depth":17,"bounds":{"left":0.09009308,"top":0.24581006,"width":0.013796543,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Drafts","depth":18,"bounds":{"left":0.09009308,"top":0.24581006,"width":0.013796543,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"8","depth":17,"bounds":{"left":0.14378324,"top":0.24700718,"width":0.0021609042,"height":0.012370312},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Purchases has menu","depth":17,"bounds":{"left":0.09009308,"top":0.26496407,"width":0.021941489,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Purchases","depth":18,"bounds":{"left":0.09009308,"top":0.26496407,"width":0.021941489,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Social 5224 unread has menu","depth":17,"bounds":{"left":0.09009308,"top":0.28411812,"width":0.013796543,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Social","depth":18,"bounds":{"left":0.09009308,"top":0.28411812,"width":0.013796543,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5,224","depth":17,"bounds":{"left":0.13613696,"top":0.28531525,"width":0.009807181,"height":0.012370312},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Updates 8834 unread has menu","depth":17,"bounds":{"left":0.09009308,"top":0.30327216,"width":0.018949468,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Updates","depth":18,"bounds":{"left":0.09009308,"top":0.30327216,"width":0.018949468,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"8,834","depth":17,"bounds":{"left":0.13613696,"top":0.3044693,"width":0.009807181,"height":0.012370312},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Forums 6161 unread has menu","depth":17,"bounds":{"left":0.09009308,"top":0.32242617,"width":0.016788565,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Forums","depth":18,"bounds":{"left":0.09009308,"top":0.32242617,"width":0.016788565,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"6,161","depth":17,"bounds":{"left":0.13746676,"top":0.3236233,"width":0.008477394,"height":0.012370312},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Promotions 38854 unread has menu","depth":17,"bounds":{"left":0.09009308,"top":0.3415802,"width":0.025930852,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Promotions","depth":18,"bounds":{"left":0.09009308,"top":0.3415802,"width":0.025930852,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"38,854","depth":17,"bounds":{"left":0.13397606,"top":0.34277734,"width":0.011968086,"height":0.012370312},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"More labels","depth":13,"bounds":{"left":0.06881649,"top":0.35834,"width":0.07978723,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"More","depth":15,"bounds":{"left":0.09009308,"top":0.36073422,"width":0.010804521,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Labels","depth":12,"bounds":{"left":0.0774601,"top":0.39984038,"width":0.061835106,"height":0.016360734},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Labels","depth":13,"bounds":{"left":0.0774601,"top":0.39984038,"width":0.016456118,"height":0.016360734},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Create new label","depth":12,"bounds":{"left":0.1392952,"top":0.40023944,"width":0.0066489363,"height":0.015961692},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Labels","depth":13,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Labels","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"[Imap]/Nevyžiadaná pošta has menu","depth":17,"bounds":{"left":0.09009308,"top":0.42817238,"width":0.055352394,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"[Imap]/Nevyžiadaná pošta","depth":18,"bounds":{"left":0.09009308,"top":0.42817238,"width":0.055352394,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"arch has menu","depth":17,"bounds":{"left":0.09009308,"top":0.44732642,"width":0.009474734,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"arch","depth":18,"bounds":{"left":0.09009308,"top":0.44732642,"width":0.009474734,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Deleted Items has menu","depth":17,"bounds":{"left":0.09009308,"top":0.46648043,"width":0.029089095,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Deleted Items","depth":18,"bounds":{"left":0.09009308,"top":0.46648043,"width":0.029089095,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Fibank 1229 unread has menu","depth":17,"bounds":{"left":0.09009308,"top":0.48563448,"width":0.01512633,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Fibank","depth":18,"bounds":{"left":0.09009308,"top":0.48563448,"width":0.01512633,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"1,229","depth":17,"bounds":{"left":0.13680187,"top":0.4868316,"width":0.009142287,"height":0.012370312},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"FL 6 unread has menu","depth":17,"bounds":{"left":0.09009308,"top":0.5047885,"width":0.005319149,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"FL","depth":18,"bounds":{"left":0.09009308,"top":0.5047885,"width":0.005319149,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"6","depth":17,"bounds":{"left":0.14361702,"top":0.5059856,"width":0.0023271276,"height":0.012370312},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Hardware & Software has menu","depth":17,"bounds":{"left":0.09009308,"top":0.52394253,"width":0.044714097,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Hardware & Software","depth":18,"bounds":{"left":0.09009308,"top":0.52394253,"width":0.044714097,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"HOSTING 5 unread has menu","depth":17,"bounds":{"left":0.09009308,"top":0.54309654,"width":0.02144282,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"HOSTING","depth":18,"bounds":{"left":0.09009308,"top":0.54309654,"width":0.02144282,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5","depth":17,"bounds":{"left":0.14378324,"top":0.5442937,"width":0.0021609042,"height":0.012370312},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Infected Items has menu","depth":17,"bounds":{"left":0.09009308,"top":0.5622506,"width":0.030086435,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Infected Items","depth":18,"bounds":{"left":0.09009308,"top":0.5622506,"width":0.030086435,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"jiminny-github 7560 unread has menu","depth":18,"bounds":{"left":0.09009308,"top":0.5814046,"width":0.03324468,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"jiminny-github","depth":19,"bounds":{"left":0.09009308,"top":0.5814046,"width":0.03324468,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"7,560","depth":18,"bounds":{"left":0.13613696,"top":0.5826017,"width":0.009807181,"height":0.012370312},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Junk E-mail 219 unread has menu","depth":17,"bounds":{"left":0.09009308,"top":0.60055864,"width":0.026761968,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Junk E-mail","depth":18,"bounds":{"left":0.09009308,"top":0.60055864,"width":0.026761968,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"219","depth":17,"bounds":{"left":0.13996011,"top":0.6017558,"width":0.005984043,"height":0.012370312},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Kontakty has menu","depth":17,"bounds":{"left":0.09009308,"top":0.6197127,"width":0.018450798,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Kontakty","depth":18,"bounds":{"left":0.09009308,"top":0.6197127,"width":0.018450798,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Sent Items has menu","depth":17,"bounds":{"left":0.09009308,"top":0.6388667,"width":0.022273935,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Sent Items","depth":18,"bounds":{"left":0.09009308,"top":0.6388667,"width":0.022273935,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"WORK 848 unread has menu","depth":17,"bounds":{"left":0.09009308,"top":0.65802073,"width":0.014461436,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"WORK","depth":18,"bounds":{"left":0.09009308,"top":0.65802073,"width":0.014461436,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"848","depth":17,"bounds":{"left":0.1392952,"top":0.6592179,"width":0.0066489363,"height":0.012370312},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"z centra 1274 unread has menu","depth":17,"bounds":{"left":0.09009308,"top":0.6771748,"width":0.018118352,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"z centra","depth":18,"bounds":{"left":0.09009308,"top":0.6771748,"width":0.018118352,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"1,274","depth":17,"bounds":{"left":0.13713431,"top":0.6783719,"width":0.00880984,"height":0.012370312},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"More labels","depth":13,"bounds":{"left":0.06881649,"top":0.69393456,"width":0.07978723,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"More","depth":15,"bounds":{"left":0.09009308,"top":0.6963288,"width":0.010804521,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXMenuButton","text":"Select","depth":12,"bounds":{"left":0.15924202,"top":0.11412609,"width":0.013297873,"height":0.015961692},"on_screen":true,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Refresh","depth":12,"bounds":{"left":0.17918883,"top":0.11412609,"width":0.0066489363,"height":0.015961692},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"More email options","depth":12,"bounds":{"left":0.1924867,"top":0.11412609,"width":0.0066489363,"height":0.015961692},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Show more messages","depth":13,"bounds":{"left":0.61702126,"top":0.10774142,"width":0.03125,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXStaticText","text":"1","depth":15,"bounds":{"left":0.6196808,"top":0.11612131,"width":0.0016622341,"height":0.012370312},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"–","depth":15,"bounds":{"left":0.6213431,"top":0.11612131,"width":0.002493351,"height":0.012370312},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"50","depth":15,"bounds":{"left":0.62383646,"top":0.11612131,"width":0.0048204786,"height":0.012370312},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"of","depth":15,"bounds":{"left":0.6286569,"top":0.11612131,"width":0.0056515955,"height":0.012370312},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"21,333","depth":15,"bounds":{"left":0.6343085,"top":0.11612131,"width":0.011303191,"height":0.012370312},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Newer","depth":13,"bounds":{"left":0.65226066,"top":0.11412609,"width":0.0066489363,"height":0.015961692},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":false,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Older","depth":13,"bounds":{"left":0.6655585,"top":0.11412609,"width":0.0066489363,"height":0.015961692},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Toggle split pane mode","depth":12,"bounds":{"left":0.6788564,"top":0.11412609,"width":0.0066489363,"height":0.015961692},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Input tools on/off (Ctrl-Shift-K)","depth":14,"bounds":{"left":0.6974734,"top":0.11412609,"width":0.0066489363,"height":0.015961692},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXMenuButton","text":"Select input tool","depth":14,"bounds":{"left":0.70412236,"top":0.11412609,"width":0.0066489363,"height":0.015961692},"on_screen":true,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Conversations","depth":13,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Conversations","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Primary","depth":15,"bounds":{"left":0.15392287,"top":0.14126097,"width":0.084109046,"height":0.03830806},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Primary","depth":16,"bounds":{"left":0.15392287,"top":0.14126097,"width":0.08344415,"height":0.03830806},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"Primary","depth":19,"bounds":{"left":0.17121011,"top":0.1528332,"width":0.016954787,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Promotions, 50 new messages,","depth":15,"bounds":{"left":0.23803191,"top":0.14126097,"width":0.084109046,"height":0.03830806},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Promotions, 50 new messages,","depth":16,"bounds":{"left":0.23803191,"top":0.14126097,"width":0.08377659,"height":0.03830806},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"50 new","depth":19,"bounds":{"left":0.28424203,"top":0.15403032,"width":0.013630319,"height":0.012370312},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Promotions","depth":19,"bounds":{"left":0.25531915,"top":0.1528332,"width":0.024933511,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Social, 18 new messages,","depth":15,"bounds":{"left":0.32214096,"top":0.14126097,"width":0.084109046,"height":0.03830806},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Social, 18 new messages,","depth":16,"bounds":{"left":0.32214096,"top":0.14126097,"width":0.08377659,"height":0.03830806},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"18 new","depth":19,"bounds":{"left":0.3565492,"top":0.15403032,"width":0.012799202,"height":0.012370312},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Social","depth":19,"bounds":{"left":0.3394282,"top":0.1528332,"width":0.013131649,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Updates","depth":15,"bounds":{"left":0.40625,"top":0.14126097,"width":0.084109046,"height":0.03830806},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Updates","depth":16,"bounds":{"left":0.40625,"top":0.14126097,"width":0.08377659,"height":0.03830806},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Updates","depth":19,"bounds":{"left":0.42353722,"top":0.1528332,"width":0.018284574,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"CircleCI Builds, [CircleCI] Workflow canceled: jiminny / app on JY-20676-delete-report-related-objects, 17:06, app / build_accept_deploy jiminny / app 2026-05-20T13:56:47Z JY-20676-delete-report-related-objects #58634 Commit push by Lukas Kovalik (LakyLak) 4f15440 JY-20676 fix tests Authored by Lukas Kovalik (.","depth":18,"bounds":{"left":0.15924202,"top":0.18276137,"width":0.0066489363,"height":0.015961692},"on_screen":true,"help_text":"","role_description":"checkbox","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Not starred","depth":18,"bounds":{"left":0.16921543,"top":0.18276137,"width":0.0066489363,"height":0.015961692},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Important mainly because it was sent directly to you.","depth":18,"bounds":{"left":0.17918883,"top":0.18276137,"width":0.0066489363,"height":0.015961692},"on_screen":true,"help_text":"","role_description":"switch","subrole":"AXSwitch","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"CircleCI Builds","depth":20,"bounds":{"left":0.18916224,"top":0.18355946,"width":0.030418882,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"[CircleCI] Workflow canceled: jiminny / app on JY-20676-delete-report-related-objects - app / build_accept_deploy jiminny / app 2026-05-20T13:56:47Z JY-20676-delete-report-related-objects #58634 Commit push by Lukas Kovalik (LakyLak) 4f15440 JY-20676 fix tests Authored by Lukas Kovalik (","depth":18,"bounds":{"left":0.2556516,"top":0.18276137,"width":0.42918882,"height":0.015961692},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"[CircleCI] Workflow canceled: jiminny / app on JY-20676-delete-report-related-objects","depth":22,"bounds":{"left":0.2556516,"top":0.18355946,"width":0.18301196,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"-","depth":22,"bounds":{"left":0.43866357,"top":0.18355946,"width":0.004488032,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"app / build_accept_deploy jiminny / app 2026-05-20T13:56:47Z JY-20676-delete-report-related-objects #58634 Commit push by Lukas Kovalik (LakyLak) 4f15440 JY-20676 fix tests Authored by Lukas Kovalik (","depth":21,"bounds":{"left":0.4431516,"top":0.18355946,"width":0.43916222,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"17:06","depth":19,"bounds":{"left":0.70644945,"top":0.18475658,"width":0.009640957,"height":0.012370312},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"CircleCI Builds, [CircleCI] Workflow failed: jiminny / app on JY-20676-delete-report-related-objects, 15:18, app / build_accept_deploy jiminny / app 2026-05-20T12:03:29Z JY-20676-delete-report-related-objects #58619 Commit push by Lukas Kovalik (LakyLak) 84a3b9c JY-20676 code review suggestions Authored by.","depth":18,"bounds":{"left":0.15924202,"top":0.20510775,"width":0.0066489363,"height":0.015961692},"on_screen":true,"help_text":"","role_description":"checkbox","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Not starred","depth":18,"bounds":{"left":0.16921543,"top":0.20510775,"width":0.0066489363,"height":0.015961692},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Important mainly because it was sent directly to you.","depth":18,"bounds":{"left":0.17918883,"top":0.20510775,"width":0.0066489363,"height":0.015961692},"on_screen":true,"help_text":"","role_description":"switch","subrole":"AXSwitch","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"CircleCI Builds","depth":20,"bounds":{"left":0.18916224,"top":0.20590582,"width":0.030418882,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"[CircleCI] Workflow failed: jiminny / app on JY-20676-delete-report-related-objects - app / build_accept_deploy jiminny / app 2026-05-20T12:03:29Z JY-20676-delete-report-related-objects #58619 Commit push by Lukas Kovalik (LakyLak) 84a3b9c JY-20676 code review suggestions Authored by","depth":18,"bounds":{"left":0.2556516,"top":0.20510775,"width":0.42918882,"height":0.015961692},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"[CircleCI] Workflow failed: jiminny / app on JY-20676-delete-report-related-objects","depth":22,"bounds":{"left":0.2556516,"top":0.20590582,"width":0.17553191,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"-","depth":22,"bounds":{"left":0.43118352,"top":0.20590582,"width":0.004488032,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"app / build_accept_deploy jiminny / app 2026-05-20T12:03:29Z JY-20676-delete-report-related-objects #58619 Commit push by Lukas Kovalik (LakyLak) 84a3b9c JY-20676 code review suggestions Authored by","depth":21,"bounds":{"left":0.43567154,"top":0.20590582,"width":0.4431516,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"15:18","depth":19,"bounds":{"left":0.7072806,"top":0.20710295,"width":0.00880984,"height":0.012370312},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"CircleCI Builds, [CircleCI] Workflow canceled: jiminny / app on JY-20613-allow-owner-role-on-team-setup, 14:44, app / build_accept_deploy jiminny / app 2026-05-20T11:44:29Z JY-20613-allow-owner-role-on-team-setup #58611 Commit push by Lukas Kovalik (LakyLak) 3642c93 JY-20613 remove recoder and voice role from.","depth":18,"bounds":{"left":0.15924202,"top":0.22745411,"width":0.0066489363,"height":0.015961692},"on_screen":true,"help_text":"","role_description":"checkbox","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Not starred","depth":18,"bounds":{"left":0.16921543,"top":0.22745411,"width":0.0066489363,"height":0.015961692},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Important mainly because it was sent directly to you.","depth":18,"bounds":{"left":0.17918883,"top":0.22745411,"width":0.0066489363,"height":0.015961692},"on_screen":true,"help_text":"","role_description":"switch","subrole":"AXSwitch","is_enabled":true,"is_focused":false,"is_selected":false}]...
|
1459673171499926460
|
6698002652732559085
|
visual_change
|
accessibility
|
NULL
|
Inbox - [EMAIL] - Gmail
Inbox - [EMAIL] - Gmail
Cl Inbox - [EMAIL] - Gmail
Inbox - [EMAIL] - Gmail
Close tab
DXP4800PLUS-B5F8
DXP4800PLUS-B5F8
(191) TED LASSO BEST SCENES - PART 2. The Most Heartwarming Ted Lasso Show Moments. - YouTube
(191) TED LASSO BEST SCENES - PART 2. The Most Heartwarming Ted Lasso Show Moments. - YouTube
LinkedIn Login, Sign in | LinkedIn
LinkedIn Login, Sign in | LinkedIn
Ambulatory sheet - Paperless-ngx
Ambulatory sheet - Paperless-ngx
New Tab
New Tab
💬1 - Apple MacBook Pro 14" Space Black M5 Max | Laptop.bg - Технологията с теб
💬1 - Apple MacBook Pro 14" Space Black M5 Max | Laptop.bg - Технологията с теб
Apple Mac Studio M4 Max | Desktop.bg - Мощни PC Гейминг конфигурации, Персонални компютри на изплащане за игри и работа
Apple Mac Studio M4 Max | Desktop.bg - Мощни PC Гейминг конфигурации, Персонални компютри на изплащане за игри и работа
Welcome to authentik! - authentik
Welcome to authentik! - authentik
Завеждане на щета онлайн | Euroins
Завеждане на щета онлайн | Euroins
New Tab
Customize sidebar
Open Google Gemini (⌃X)
Open history (⇧⌘H)
Open bookmarks (⌘B)
Bitwarden
None selected
Skip to content
Skip to content
Using Gmail with screen readers
Using Gmail with screen readers
Main menu
Gmail
Search
Search
Search mail
Advanced search options
Search mail
Support
Settings
Ask Gemini
Google apps
Google Account: Lukáš Koválik ([EMAIL])
Compose
Labels
Labels
Inbox
Inbox
Starred
Starred
Snoozed
Snoozed
Important
Important
Sent
Sent
Drafts 8 unread
Drafts
8
Purchases has menu
Purchases
Social 5224 unread has menu
Social
5,224
Updates 8834 unread has menu
Updates
8,834
Forums 6161 unread has menu
Forums
6,161
Promotions 38854 unread has menu
Promotions
38,854
More labels
More
Labels
Labels
Create new label
Labels
Labels
[Imap]/Nevyžiadaná pošta has menu
[Imap]/Nevyžiadaná pošta
arch has menu
arch
Deleted Items has menu
Deleted Items
Fibank 1229 unread has menu
Fibank
1,229
FL 6 unread has menu
FL
6
Hardware & Software has menu
Hardware & Software
HOSTING 5 unread has menu
HOSTING
5
Infected Items has menu
Infected Items
jiminny-github 7560 unread has menu
jiminny-github
7,560
Junk E-mail 219 unread has menu
Junk E-mail
219
Kontakty has menu
Kontakty
Sent Items has menu
Sent Items
WORK 848 unread has menu
WORK
848
z centra 1274 unread has menu
z centra
1,274
More labels
More
Select
Refresh
More email options
Show more messages
1
–
50
of
21,333
Newer
Older
Toggle split pane mode
Input tools on/off (Ctrl-Shift-K)
Select input tool
Conversations
Conversations
Primary
Primary
Primary
Promotions, 50 new messages,
Promotions, 50 new messages,
50 new
Promotions
Social, 18 new messages,
Social, 18 new messages,
18 new
Social
Updates
Updates
Updates
CircleCI Builds, [CircleCI] Workflow canceled: jiminny / app on JY-20676-delete-report-related-objects, 17:06, app / build_accept_deploy jiminny / app 2026-05-20T13:56:47Z JY-20676-delete-report-related-objects #58634 Commit push by Lukas Kovalik (LakyLak) 4f15440 JY-20676 fix tests Authored by Lukas Kovalik (.
Not starred
Important mainly because it was sent directly to you.
CircleCI Builds
[CircleCI] Workflow canceled: jiminny / app on JY-20676-delete-report-related-objects - app / build_accept_deploy jiminny / app 2026-05-20T13:56:47Z JY-20676-delete-report-related-objects #58634 Commit push by Lukas Kovalik (LakyLak) 4f15440 JY-20676 fix tests Authored by Lukas Kovalik (
[CircleCI] Workflow canceled: jiminny / app on JY-20676-delete-report-related-objects
-
app / build_accept_deploy jiminny / app 2026-05-20T13:56:47Z JY-20676-delete-report-related-objects #58634 Commit push by Lukas Kovalik (LakyLak) 4f15440 JY-20676 fix tests Authored by Lukas Kovalik (
17:06
CircleCI Builds, [CircleCI] Workflow failed: jiminny / app on JY-20676-delete-report-related-objects, 15:18, app / build_accept_deploy jiminny / app 2026-05-20T12:03:29Z JY-20676-delete-report-related-objects #58619 Commit push by Lukas Kovalik (LakyLak) 84a3b9c JY-20676 code review suggestions Authored by.
Not starred
Important mainly because it was sent directly to you.
CircleCI Builds
[CircleCI] Workflow failed: jiminny / app on JY-20676-delete-report-related-objects - app / build_accept_deploy jiminny / app 2026-05-20T12:03:29Z JY-20676-delete-report-related-objects #58619 Commit push by Lukas Kovalik (LakyLak) 84a3b9c JY-20676 code review suggestions Authored by
[CircleCI] Workflow failed: jiminny / app on JY-20676-delete-report-related-objects
-
app / build_accept_deploy jiminny / app 2026-05-20T12:03:29Z JY-20676-delete-report-related-objects #58619 Commit push by Lukas Kovalik (LakyLak) 84a3b9c JY-20676 code review suggestions Authored by
15:18
CircleCI Builds, [CircleCI] Workflow canceled: jiminny / app on JY-20613-allow-owner-role-on-team-setup, 14:44, app / build_accept_deploy jiminny / app 2026-05-20T11:44:29Z JY-20613-allow-owner-role-on-team-setup #58611 Commit push by Lukas Kovalik (LakyLak) 3642c93 JY-20613 remove recoder and voice role from.
Not starred
Important mainly because it was sent directly to you....
|
65076
|
NULL
|
NULL
|
NULL
|
|
65082
|
2280
|
8
|
2026-05-20T15:09:59.990353+00:00
|
/Users/lukas/.screenpipe/data/data/2026-05-20/1779 /Users/lukas/.screenpipe/data/data/2026-05-20/1779289799990_m2.jpg...
|
Firefox
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira — Work...
|
1
|
jiminny.atlassian.net/jira/software/c/projects/JY/ jiminny.atlassian.net/jira/software/c/projects/JY/boards/37...
|
monitor_2
|
NULL
|
NULL
|
NULL
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Close tab
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {"status":"error","message":"You have reached your secondly limit.","errorType":"RATE_LIMIT
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {"status":"error","message":"You have reached your secondly limit.","errorType":"RATE_LIMIT
Pipelines - jiminny/app
Pipelines - jiminny/app
Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app
Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app
New Tab
Customize sidebar
Close Google Gemini (⌃X)
Tabs from other devices
Open history (⇧⌘H)
Open bookmarks (⌘B)
AI Chat settings
Close...
|
[{"role":"AXRadioButton","text [{"role":"AXRadioButton","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":4,"bounds":{"left":0.0,"top":0.0518755,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.06304868,"width":0.10106383,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Close tab","depth":5,"bounds":{"left":0.06732048,"top":0.05905826,"width":0.007978723,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXRadioButton","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":4,"bounds":{"left":0.0,"top":0.08459697,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.09577015,"width":0.10721409,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.11731844,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.12849163,"width":0.15791224,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.15003991,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.16121309,"width":0.1796875,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.18276137,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.19393456,"width":0.12699468,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":4,"bounds":{"left":0.0,"top":0.21548285,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":5,"bounds":{"left":0.013297873,"top":0.22665602,"width":0.10139628,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":4,"bounds":{"left":0.0,"top":0.2482043,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.25937748,"width":0.18068483,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":4,"bounds":{"left":0.0,"top":0.28092578,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":5,"bounds":{"left":0.013297873,"top":0.29209897,"width":0.4644282,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Pipelines - jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.31364724,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Pipelines - jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.32482043,"width":0.039228722,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.3463687,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.3575419,"width":0.15824468,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Tab","depth":4,"bounds":{"left":0.0028257978,"top":0.38068634,"width":0.07413564,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Customize sidebar","depth":6,"bounds":{"left":0.0028257978,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Close Google Gemini (⌃X)","depth":6,"bounds":{"left":0.013796543,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Tabs from other devices","depth":6,"bounds":{"left":0.024933511,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open history (⇧⌘H)","depth":6,"bounds":{"left":0.036070477,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open bookmarks (⌘B)","depth":6,"bounds":{"left":0.04720745,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"AI Chat settings","depth":7,"bounds":{"left":0.35854387,"top":0.055067837,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Close","depth":7,"bounds":{"left":0.37051198,"top":0.055067837,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false}]...
|
429655369988079005
|
-1132805328839085433
|
app_switch
|
accessibility
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Close tab
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {"status":"error","message":"You have reached your secondly limit.","errorType":"RATE_LIMIT
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {"status":"error","message":"You have reached your secondly limit.","errorType":"RATE_LIMIT
Pipelines - jiminny/app
Pipelines - jiminny/app
Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app
Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app
New Tab
Customize sidebar
Close Google Gemini (⌃X)
Tabs from other devices
Open history (⇧⌘H)
Open bookmarks (⌘B)
AI Chat settings
Close...
|
NULL
|
NULL
|
NULL
|
NULL
|
|
65083
|
2279
|
7
|
2026-05-20T15:09:59.197890+00:00
|
/Users/lukas/.screenpipe/data/data/2026-05-20/1779 /Users/lukas/.screenpipe/data/data/2026-05-20/1779289799197_m1.jpg...
|
Firefox
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira — Work...
|
1
|
jiminny.atlassian.net/jira/software/c/projects/JY/ jiminny.atlassian.net/jira/software/c/projects/JY/boards/37...
|
monitor_1
|
NULL
|
NULL
|
NULL
|
NULL
|
SlackFileEditViewGoHistoryWindowHelp> 0 lll20A1 SlackFileEditViewGoHistoryWindowHelp> 0 lll20A100% C 8• Wed 20 May 18:09:59DOCKERO 81DEV (-zsh)APP (-zsh)|₴2181screenpipe"• ₴3APP (-zsh)Fixed 0 of 5690 files in 78.144 seconds, 60.00MBmemoryusedWhat's next:Try Docker Debug for seamless, persistentdebugging tools in any containeror image → docker debug docker_lamp_1Learn more at [URL_WITH_CREDENTIALS] (JY-20676-delete-report-related-objects) $ sips -s format png ~/Downloads/Photos-3-001\(2\)/talon-f.HEIC --out ~/Downloads/Photos-3-001\(2\)/talon-f.png/Users/lukas/Downloads/Photos-3-001(2)/talon-f.HEIC/Users/lukas/Downloads/Photos-3-001(2)/talon-f.pnglukas@Lukas-Kovaliks-MacBook-Pro-Jiminny ~/jiminny/app(JY-20676-delete-report-related-objects) $ sips -s format png ~/Downloads/Photos-3-001\(2\)/talon-b.HEIC --out ~/Downloads/Photos-3-001\(2\)/talon-b.png/Users/lukas/Downloads/Photos-3-001(2)/talon-b.HEIC/Users/lukas/Downloads/Photos-3-001(2)/talon-b.pnglukas@Lukas-Kovaliks-MacBook-Pro-Jiminny ~/jiminny/app(JY-20676-delete-report-related-objects) $ sips -s format png ~/Downloads/Photos-3-001\(2\)/vin.HEIC --out ~/Downloads/Photos-3-001\(2\)/vin.png/Users/lukas/Downloads/Photos-3-001(2)/vin.HEIC/Users/lukas/Downloads/Photos-3-001(2)/vin.pnglukas@Lukas-Kovaliks-MacBook-Pro-Jiminny ~/jiminny/app (JY-20676-delete-report-related-objects) $ sips -s format png ~/Downloads/Photos-3-001\(2\)/dr-lic-f.HEIC --out ~/Downloads/Photos-3-001\(2\)/dr-lic-f.png/Users/lukas/Downloads/Photos-3-001(2)/dr-lic-f.HEIC/Users/lukas/Downloads/Photos-3-001(2)/dr-lic-f.pnglukas@Lukas-Kovaliks-MacBook-Pro-Jiminny ~/jiminny/app (JY-20676-delete-report-related-objects) $ sips -s format png ~/Downloads/Photos-3-001\(2\)/dr-lic-b.HEIC --out ~/Downloads/Photos-3-001\(2\)/dr-lic-b.png/Users/lukas/Downloads/Photos-3-001(2)/dr-lic-b.HEIC/Users/lukas/Downloads/Photos-3-001(2)/dr-lic-b.pnglukas@Lukas-Kovaliks-MacBook-Pro-Jiminny ~/jiminny/app (JY-20676-delete-report-related-objects)sips -s format png ~/Downloads/Photos-3-001\(2\)/gtp.HEIC --out ~/Downloads/Photos-3-001\(2)/gtp.pngWarning: /Users/lukas/Downloads/Photos-3-001(2)/gtp.HEIC not a valid file - skippinglukas@Lukas-Kovaliks-MacBook-Pro-Jiminny ~/jiminny/app (JY-20676-delete-report-related-objects) $ sips -s format png ~/Downloads/Photos-3-001\(2\)/gtp.heif --out ~/Downloads/Photos-3-001\(2\)/gtp.png/Users/lukas/Downloads/Photos-3-001(2)/gtp.heif/Users/lukas/Downloads/Photos-3-001(2)/gtp.pnglukas@Lukas-Kovaliks-MacBook-Pro-Jiminny ~/jiminny/app (JY-20676-delete-report-related-objects) $ sips -s format png ~/Downloads/Photos-3-001\(2\)/talon-m-f.HEIC --out ~/Downloads/Photos-3-001\(2\)/talon-m-f.png/Users/lukas/Downloads/Photos-3-001(2)/talon-m-f.HEIC/Users/lukas/Downloads/Photos-3-001(2)/talon-m-f.pnglukas@Lukas-Kovaliks-MacBook-Pro-Jiminny ~/Jiminny/app(JY-20676-delete-report-related-objects) $ sips -s format png ~/Downloads/Photos-3-001\(2\)/talon-m-b.HEIC --out ~/Downloads/Photos-3-001\(2\)/talon-m-b.png/Users/lukas/Downloads/Photos-3-001(2)/talon-m-b.HEIC/Users/lukas/Downloads/Photos-3-001(2)/talon-m-b.pnglukas@Lukas-Kovaliks-MacBook-Pro-Jiminny ~/jiminny/app (JY-20676-delete-report-related-objects) $D...
|
NULL
|
-736909638546705292
|
NULL
|
visual_change
|
ocr
|
NULL
|
SlackFileEditViewGoHistoryWindowHelp> 0 lll20A1 SlackFileEditViewGoHistoryWindowHelp> 0 lll20A100% C 8• Wed 20 May 18:09:59DOCKERO 81DEV (-zsh)APP (-zsh)|₴2181screenpipe"• ₴3APP (-zsh)Fixed 0 of 5690 files in 78.144 seconds, 60.00MBmemoryusedWhat's next:Try Docker Debug for seamless, persistentdebugging tools in any containeror image → docker debug docker_lamp_1Learn more at [URL_WITH_CREDENTIALS] (JY-20676-delete-report-related-objects) $ sips -s format png ~/Downloads/Photos-3-001\(2\)/talon-f.HEIC --out ~/Downloads/Photos-3-001\(2\)/talon-f.png/Users/lukas/Downloads/Photos-3-001(2)/talon-f.HEIC/Users/lukas/Downloads/Photos-3-001(2)/talon-f.pnglukas@Lukas-Kovaliks-MacBook-Pro-Jiminny ~/jiminny/app(JY-20676-delete-report-related-objects) $ sips -s format png ~/Downloads/Photos-3-001\(2\)/talon-b.HEIC --out ~/Downloads/Photos-3-001\(2\)/talon-b.png/Users/lukas/Downloads/Photos-3-001(2)/talon-b.HEIC/Users/lukas/Downloads/Photos-3-001(2)/talon-b.pnglukas@Lukas-Kovaliks-MacBook-Pro-Jiminny ~/jiminny/app(JY-20676-delete-report-related-objects) $ sips -s format png ~/Downloads/Photos-3-001\(2\)/vin.HEIC --out ~/Downloads/Photos-3-001\(2\)/vin.png/Users/lukas/Downloads/Photos-3-001(2)/vin.HEIC/Users/lukas/Downloads/Photos-3-001(2)/vin.pnglukas@Lukas-Kovaliks-MacBook-Pro-Jiminny ~/jiminny/app (JY-20676-delete-report-related-objects) $ sips -s format png ~/Downloads/Photos-3-001\(2\)/dr-lic-f.HEIC --out ~/Downloads/Photos-3-001\(2\)/dr-lic-f.png/Users/lukas/Downloads/Photos-3-001(2)/dr-lic-f.HEIC/Users/lukas/Downloads/Photos-3-001(2)/dr-lic-f.pnglukas@Lukas-Kovaliks-MacBook-Pro-Jiminny ~/jiminny/app (JY-20676-delete-report-related-objects) $ sips -s format png ~/Downloads/Photos-3-001\(2\)/dr-lic-b.HEIC --out ~/Downloads/Photos-3-001\(2\)/dr-lic-b.png/Users/lukas/Downloads/Photos-3-001(2)/dr-lic-b.HEIC/Users/lukas/Downloads/Photos-3-001(2)/dr-lic-b.pnglukas@Lukas-Kovaliks-MacBook-Pro-Jiminny ~/jiminny/app (JY-20676-delete-report-related-objects)sips -s format png ~/Downloads/Photos-3-001\(2\)/gtp.HEIC --out ~/Downloads/Photos-3-001\(2)/gtp.pngWarning: /Users/lukas/Downloads/Photos-3-001(2)/gtp.HEIC not a valid file - skippinglukas@Lukas-Kovaliks-MacBook-Pro-Jiminny ~/jiminny/app (JY-20676-delete-report-related-objects) $ sips -s format png ~/Downloads/Photos-3-001\(2\)/gtp.heif --out ~/Downloads/Photos-3-001\(2\)/gtp.png/Users/lukas/Downloads/Photos-3-001(2)/gtp.heif/Users/lukas/Downloads/Photos-3-001(2)/gtp.pnglukas@Lukas-Kovaliks-MacBook-Pro-Jiminny ~/jiminny/app (JY-20676-delete-report-related-objects) $ sips -s format png ~/Downloads/Photos-3-001\(2\)/talon-m-f.HEIC --out ~/Downloads/Photos-3-001\(2\)/talon-m-f.png/Users/lukas/Downloads/Photos-3-001(2)/talon-m-f.HEIC/Users/lukas/Downloads/Photos-3-001(2)/talon-m-f.pnglukas@Lukas-Kovaliks-MacBook-Pro-Jiminny ~/Jiminny/app(JY-20676-delete-report-related-objects) $ sips -s format png ~/Downloads/Photos-3-001\(2\)/talon-m-b.HEIC --out ~/Downloads/Photos-3-001\(2\)/talon-m-b.png/Users/lukas/Downloads/Photos-3-001(2)/talon-m-b.HEIC/Users/lukas/Downloads/Photos-3-001(2)/talon-m-b.pnglukas@Lukas-Kovaliks-MacBook-Pro-Jiminny ~/jiminny/app (JY-20676-delete-report-related-objects) $D...
|
65081
|
NULL
|
NULL
|
NULL
|
|
65084
|
2279
|
8
|
2026-05-20T15:10:02.215959+00:00
|
/Users/lukas/.screenpipe/data/data/2026-05-20/1779 /Users/lukas/.screenpipe/data/data/2026-05-20/1779289802215_m1.jpg...
|
Firefox
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira — Work...
|
1
|
jiminny.atlassian.net/jira/software/c/projects/JY/ jiminny.atlassian.net/jira/software/c/projects/JY/boards/37...
|
monitor_1
|
NULL
|
NULL
|
NULL
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Close tab
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No...
|
[{"role":"AXRadioButton","text [{"role":"AXRadioButton","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Close tab","depth":5,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXRadioButton","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Pipelines - jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Pipelines - jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Tab","depth":4,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Customize sidebar","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Close Google Gemini (⌃X)","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Tabs from other devices","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open history (⇧⌘H)","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open bookmarks (⌘B)","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"AI Chat settings","depth":7,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Close","depth":7,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Main menu","depth":13,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"New chat","depth":12,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Gemini","depth":15,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Chat","depth":13,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Open menu for conversation actions.","depth":13,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Conversation with Gemini","depth":15,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Conversation with Gemini","depth":16,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"You said I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.","depth":23,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Expand","depth":21,"on_screen":false,"role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a concise summary of the changes made in the code selection:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Refactoring and Dependencies","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Refactoring and Dependencies","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Replaced Interface with Implementation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Changed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusInterface","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusManager","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"in the imports and constructor dependency injection.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Redis Support:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Imported","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Illuminate\\Support\\Facades\\Redis","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to handle a newly introduced memory-logging lock mechanism.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Traits:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"WorkerIdTrait","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", along with the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"worker","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"field tracking in the final logs.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Loop and Worker Lifecycle Adjustments","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Loop and Worker Lifecycle Adjustments","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Worker Auto-Restart:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Deleted the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property, the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"constant, and the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"shouldIterateAgain()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method. The script no longer tracks execution iterations or triggers a self-managed worker restart.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Simplified Chunk Method Return:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Modified","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to always return","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Priority Handling for Throttled Entities","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Priority Handling for Throttled Entities","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Stale Record Elevation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Updated the logic in","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"rescheduleThrottledEntities()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to check if a throttled entity group is classified as \"stale\". If it is, the entities are rescheduled using","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"QueuePriorityEnum::NORMAL","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"; otherwise, they keep their respective queue priority.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Metrics, Logging, and Environment Handling","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Metrics, Logging, and Environment Handling","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Environment Tracking:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Introduced a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$environment","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property (defaulting to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"'local'","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") and a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"setEnvironment()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method to dynamically assign it.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Memory Monitoring via Datadog Gauge:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method that submits peak memory allocation to Datadog as a gauge metric (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"jiminny.reindex-memory-usage","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis Lock for Metrics Throttling:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Implemented a Redis-backed lock (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"%s-process-manager-lock","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") with a 60-second expiration inside","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to prevent over-reporting metrics.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Log Payload Update:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added total processed items (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$updatedDocuments + $deletedDocuments","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") to the final information log.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":23,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"You said are there any issues or potential bottlenecks","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"are there any issues or potential bottlenecks","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Edit","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a breakdown of the potential issues to look out for:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"1. Memory Leak Risk (OOM Crashes)","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"1. Memory Leak Risk (OOM Crashes)","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Restart Logic:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The most significant change is the removal of the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Dependency Leaks:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If any underlying service (like the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ElasticSearchClient","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", ORM, or Datadog helper) retains references to processed entities,","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Hardcoded Limits:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The code still includes","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ini_set('memory_limit', '512M');","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":". Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"2. Race Condition in Redis Lock","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"2. Race Condition in Redis Lock","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Non-Atomic Operations:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method uses three distinct steps to handle the lock:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"get()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":",","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", and","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"expire()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If multiple workers process the same","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$this->updateTarget","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"concurrently, they can both pass the","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"if (! is_null(Redis::get(...)))","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"check at the exact same millisecond. Both will then execute","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and push duplicate metrics to Datadog.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Fix:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"This should be replaced with a single atomic operation using Redis's","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"NX","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Not eXists) and","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"EX","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Expire) flags, or Laravel’s built-in atomic cache locks:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis::set($gaugeLockName, true, 'EX', 60, 'NX')","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"3. Infinite Loops and Caller Contract","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"3. Infinite Loops and Caller Contract","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Return Value Change:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"previously returned whether it should iterate again based on staleness or chunk size. It now always returns","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(unless the selection is totally empty).","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the code calling this method relies on a","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"false","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"return to pause, sleep, or gracefully terminate, changing this to an unconditional","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"could result in an aggressive \"busy loop\" that spikes CPU usage.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"4. Performance Overhead","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"4. Performance Overhead","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Expensive Garbage Collection:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Calling","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles();","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"on","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"every single chunk","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Synchronous Metric Dispatch:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::increment","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"or","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::gauge","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"5. Lack of Explicit Error Handling","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5. Lack of Explicit Error Handling","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"No","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"}]...
|
-7509020431186018364
|
-1070952245176731684
|
visual_change
|
accessibility
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Close tab
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No...
|
NULL
|
NULL
|
NULL
|
NULL
|
|
65085
|
2280
|
9
|
2026-05-20T15:10:05.775217+00:00
|
/Users/lukas/.screenpipe/data/data/2026-05-20/1779 /Users/lukas/.screenpipe/data/data/2026-05-20/1779289805775_m2.jpg...
|
Firefox
|
Inbox - kovaliklukas@gmail.com - Gmail — Personal
|
1
|
mail.google.com/mail/u/0/#inbox
|
monitor_2
|
NULL
|
NULL
|
NULL
|
NULL
|
Inbox - [EMAIL] - Gmail
Inbox - [EMAIL] - Gmail
Cl Inbox - [EMAIL] - Gmail
Inbox - [EMAIL] - Gmail
Close tab
DXP4800PLUS-B5F8
DXP4800PLUS-B5F8
(191) TED LASSO BEST SCENES - PART 2. The Most Heartwarming Ted Lasso Show Moments. - YouTube
(191) TED LASSO BEST SCENES - PART 2. The Most Heartwarming Ted Lasso Show Moments. - YouTube
LinkedIn Login, Sign in | LinkedIn
LinkedIn Login, Sign in | LinkedIn
Ambulatory sheet - Paperless-ngx
Ambulatory sheet - Paperless-ngx
New Tab
New Tab
💬1 - Apple MacBook Pro 14" Space Black M5 Max | Laptop.bg - Технологията с теб
💬1 - Apple MacBook Pro 14" Space Black M5 Max | Laptop.bg - Технологията с теб
Apple Mac Studio M4 Max | Desktop.bg - Мощни PC Гейминг конфигурации, Персонални компютри на изплащане за игри и работа
Apple Mac Studio M4 Max | Desktop.bg - Мощни PC Гейминг конфигурации, Персонални компютри на изплащане за игри и работа
Welcome to authentik! - authentik
Welcome to authentik! - authentik
Завеждане на щета онлайн | Euroins
Завеждане на щета онлайн | Euroins
New Tab
Customize sidebar
Open Google Gemini (⌃X)
Open history (⇧⌘H)
Open bookmarks (⌘B)
Bitwarden
None selected
Skip to content
Skip to content
Using Gmail with screen readers
Using Gmail with screen readers
Main menu
Gmail
Search
Search
Search mail
Advanced search options
Search mail
Support
Settings
Ask Gemini
Google apps
Google Account: Lukáš Koválik ([EMAIL])
Compose
Labels
Labels
Inbox
Inbox
Starred
Starred
Snoozed
Snoozed
Important
Important
Sent
Sent
Drafts 8 unread
Drafts
8
Purchases has menu
Purchases
Social 5224 unread has menu
Social
5,224
Updates 8834 unread has menu
Updates
8,834
Forums 6161 unread has menu
Forums
6,161
Promotions 38854 unread has menu
Promotions
38,854
More labels
More...
|
[{"role":"AXRadioButton","text [{"role":"AXRadioButton","text":"Inbox - kovaliklukas@gmail.com - Gmail","depth":4,"bounds":{"left":0.0,"top":0.0518755,"width":0.06881649,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"Inbox - kovaliklukas@gmail.com - Gmail","depth":5,"bounds":{"left":0.013297873,"top":0.06304868,"width":0.06898271,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Close tab","depth":5,"bounds":{"left":0.05651596,"top":0.05905826,"width":0.007978723,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXRadioButton","text":"DXP4800PLUS-B5F8","depth":4,"bounds":{"left":0.0,"top":0.08459697,"width":0.06881649,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"DXP4800PLUS-B5F8","depth":5,"bounds":{"left":0.013297873,"top":0.09577015,"width":0.036901597,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"(191) TED LASSO BEST SCENES - PART 2. The Most Heartwarming Ted Lasso Show Moments. - YouTube","depth":4,"bounds":{"left":0.0,"top":0.11731844,"width":0.06881649,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"(191) TED LASSO BEST SCENES - PART 2. The Most Heartwarming Ted Lasso Show Moments. - YouTube","depth":5,"bounds":{"left":0.013297873,"top":0.12849163,"width":0.18218085,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"LinkedIn Login, Sign in | LinkedIn","depth":4,"bounds":{"left":0.0,"top":0.15003991,"width":0.06881649,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LinkedIn Login, Sign in | LinkedIn","depth":5,"bounds":{"left":0.013297873,"top":0.16121309,"width":0.05668218,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Ambulatory sheet - Paperless-ngx","depth":4,"bounds":{"left":0.0,"top":0.18276137,"width":0.06881649,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Ambulatory sheet - Paperless-ngx","depth":5,"bounds":{"left":0.013297873,"top":0.19393456,"width":0.059507977,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"New Tab","depth":4,"bounds":{"left":0.0,"top":0.21548285,"width":0.06881649,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"New Tab","depth":5,"bounds":{"left":0.013297873,"top":0.22665602,"width":0.014960106,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"💬1 - Apple MacBook Pro 14\" Space Black M5 Max | Laptop.bg - Технологията с теб","depth":4,"bounds":{"left":0.0,"top":0.2482043,"width":0.06881649,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"💬1 - Apple MacBook Pro 14\" Space Black M5 Max | Laptop.bg - Технологията с теб","depth":5,"bounds":{"left":0.013297873,"top":0.25937748,"width":0.14727394,"height":0.011572227},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Apple Mac Studio M4 Max | Desktop.bg - Мощни PC Гейминг конфигурации, Персонални компютри на изплащане за игри и работа","depth":4,"bounds":{"left":0.0,"top":0.28092578,"width":0.06881649,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Apple Mac Studio M4 Max | Desktop.bg - Мощни PC Гейминг конфигурации, Персонални компютри на изплащане за игри и работа","depth":5,"bounds":{"left":0.013297873,"top":0.29209897,"width":0.2365359,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Welcome to authentik! - authentik","depth":4,"bounds":{"left":0.0,"top":0.31364724,"width":0.06881649,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Welcome to authentik! - authentik","depth":5,"bounds":{"left":0.013297873,"top":0.32482043,"width":0.05900931,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Завеждане на щета онлайн | Euroins","depth":4,"bounds":{"left":0.0,"top":0.3463687,"width":0.06881649,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Завеждане на щета онлайн | Euroins","depth":5,"bounds":{"left":0.013297873,"top":0.3575419,"width":0.0653258,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Tab","depth":4,"bounds":{"left":0.0028257978,"top":0.38068634,"width":0.06333112,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Customize sidebar","depth":6,"bounds":{"left":0.0028257978,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open Google Gemini (⌃X)","depth":6,"bounds":{"left":0.013796543,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open history (⇧⌘H)","depth":6,"bounds":{"left":0.024933511,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open bookmarks (⌘B)","depth":6,"bounds":{"left":0.036070477,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bitwarden","depth":6,"bounds":{"left":0.04720745,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"None selected","depth":8,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Skip to content","depth":10,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Skip to content","depth":11,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Using Gmail with screen readers","depth":10,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXStaticText","text":"Using Gmail with screen readers","depth":11,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Main menu","depth":11,"bounds":{"left":0.07280585,"top":0.058260176,"width":0.015957447,"height":0.03830806},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":true},{"role":"AXLink","text":"Gmail","depth":12,"bounds":{"left":0.09009308,"top":0.061452515,"width":0.036236703,"height":0.035115723},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Search","depth":12,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Search","depth":13,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXTextField","text":"Search mail","depth":18,"bounds":{"left":0.17220744,"top":0.06943336,"width":0.18916224,"height":0.016360734},"on_screen":true,"help_text":"","role_description":"text field","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Advanced search options","depth":12,"bounds":{"left":0.37466756,"top":0.058260176,"width":0.01861702,"height":0.03671189},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Search mail","depth":12,"bounds":{"left":0.15392287,"top":0.058260176,"width":0.01861702,"height":0.03671189},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":false,"is_focused":false,"is_selected":false},{"role":"AXMenuButton","text":"Support","depth":12,"bounds":{"left":0.66356385,"top":0.061452515,"width":0.013297873,"height":0.031923383},"on_screen":true,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXMenuButton","text":"Settings","depth":12,"bounds":{"left":0.6781915,"top":0.061452515,"width":0.013297873,"height":0.031923383},"on_screen":true,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Ask Gemini","depth":12,"bounds":{"left":0.6921542,"top":0.061452515,"width":0.013297873,"height":0.031923383},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Google apps","depth":14,"bounds":{"left":0.70611703,"top":0.061452515,"width":0.013297873,"height":0.031923383},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Google Account: Lukáš Koválik (kovaliklukas@gmail.com)","depth":14,"bounds":{"left":0.72207445,"top":0.061452515,"width":0.013297873,"height":0.031923383},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Compose","depth":9,"bounds":{"left":0.071476065,"top":0.10933759,"width":0.04737367,"height":0.031923383},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Labels","depth":13,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Labels","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Inbox","depth":17,"bounds":{"left":0.09009308,"top":0.15003991,"width":0.012466756,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Inbox","depth":18,"bounds":{"left":0.09009308,"top":0.15003991,"width":0.012466756,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Starred","depth":17,"bounds":{"left":0.09009308,"top":0.16919394,"width":0.015625,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Starred","depth":18,"bounds":{"left":0.09009308,"top":0.16919394,"width":0.015625,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Snoozed","depth":17,"bounds":{"left":0.09009308,"top":0.18834797,"width":0.018284574,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Snoozed","depth":18,"bounds":{"left":0.09009308,"top":0.18834797,"width":0.018284574,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Important","depth":18,"bounds":{"left":0.09009308,"top":0.207502,"width":0.020777926,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Important","depth":19,"bounds":{"left":0.09009308,"top":0.207502,"width":0.020777926,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Sent","depth":17,"bounds":{"left":0.09009308,"top":0.22665602,"width":0.009640957,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Sent","depth":18,"bounds":{"left":0.09009308,"top":0.22665602,"width":0.009640957,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Drafts 8 unread","depth":17,"bounds":{"left":0.09009308,"top":0.24581006,"width":0.013796543,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Drafts","depth":18,"bounds":{"left":0.09009308,"top":0.24581006,"width":0.013796543,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"8","depth":17,"bounds":{"left":0.14378324,"top":0.24700718,"width":0.0021609042,"height":0.012370312},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Purchases has menu","depth":17,"bounds":{"left":0.09009308,"top":0.26496407,"width":0.021941489,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Purchases","depth":18,"bounds":{"left":0.09009308,"top":0.26496407,"width":0.021941489,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Social 5224 unread has menu","depth":17,"bounds":{"left":0.09009308,"top":0.28411812,"width":0.013796543,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Social","depth":18,"bounds":{"left":0.09009308,"top":0.28411812,"width":0.013796543,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5,224","depth":17,"bounds":{"left":0.13613696,"top":0.28531525,"width":0.009807181,"height":0.012370312},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Updates 8834 unread has menu","depth":17,"bounds":{"left":0.09009308,"top":0.30327216,"width":0.018949468,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Updates","depth":18,"bounds":{"left":0.09009308,"top":0.30327216,"width":0.018949468,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"8,834","depth":17,"bounds":{"left":0.13613696,"top":0.3044693,"width":0.009807181,"height":0.012370312},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Forums 6161 unread has menu","depth":17,"bounds":{"left":0.09009308,"top":0.32242617,"width":0.016788565,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Forums","depth":18,"bounds":{"left":0.09009308,"top":0.32242617,"width":0.016788565,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"6,161","depth":17,"bounds":{"left":0.13746676,"top":0.3236233,"width":0.008477394,"height":0.012370312},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Promotions 38854 unread has menu","depth":17,"bounds":{"left":0.09009308,"top":0.3415802,"width":0.025930852,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Promotions","depth":18,"bounds":{"left":0.09009308,"top":0.3415802,"width":0.025930852,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"38,854","depth":17,"bounds":{"left":0.13397606,"top":0.34277734,"width":0.011968086,"height":0.012370312},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"More labels","depth":13,"bounds":{"left":0.06881649,"top":0.35834,"width":0.07978723,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"More","depth":15,"bounds":{"left":0.09009308,"top":0.36073422,"width":0.010804521,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"}]...
|
-1104118504249266728
|
1355614813563227309
|
app_switch
|
accessibility
|
NULL
|
Inbox - [EMAIL] - Gmail
Inbox - [EMAIL] - Gmail
Cl Inbox - [EMAIL] - Gmail
Inbox - [EMAIL] - Gmail
Close tab
DXP4800PLUS-B5F8
DXP4800PLUS-B5F8
(191) TED LASSO BEST SCENES - PART 2. The Most Heartwarming Ted Lasso Show Moments. - YouTube
(191) TED LASSO BEST SCENES - PART 2. The Most Heartwarming Ted Lasso Show Moments. - YouTube
LinkedIn Login, Sign in | LinkedIn
LinkedIn Login, Sign in | LinkedIn
Ambulatory sheet - Paperless-ngx
Ambulatory sheet - Paperless-ngx
New Tab
New Tab
💬1 - Apple MacBook Pro 14" Space Black M5 Max | Laptop.bg - Технологията с теб
💬1 - Apple MacBook Pro 14" Space Black M5 Max | Laptop.bg - Технологията с теб
Apple Mac Studio M4 Max | Desktop.bg - Мощни PC Гейминг конфигурации, Персонални компютри на изплащане за игри и работа
Apple Mac Studio M4 Max | Desktop.bg - Мощни PC Гейминг конфигурации, Персонални компютри на изплащане за игри и работа
Welcome to authentik! - authentik
Welcome to authentik! - authentik
Завеждане на щета онлайн | Euroins
Завеждане на щета онлайн | Euroins
New Tab
Customize sidebar
Open Google Gemini (⌃X)
Open history (⇧⌘H)
Open bookmarks (⌘B)
Bitwarden
None selected
Skip to content
Skip to content
Using Gmail with screen readers
Using Gmail with screen readers
Main menu
Gmail
Search
Search
Search mail
Advanced search options
Search mail
Support
Settings
Ask Gemini
Google apps
Google Account: Lukáš Koválik ([EMAIL])
Compose
Labels
Labels
Inbox
Inbox
Starred
Starred
Snoozed
Snoozed
Important
Important
Sent
Sent
Drafts 8 unread
Drafts
8
Purchases has menu
Purchases
Social 5224 unread has menu
Social
5,224
Updates 8834 unread has menu
Updates
8,834
Forums 6161 unread has menu
Forums
6,161
Promotions 38854 unread has menu
Promotions
38,854
More labels
More...
|
NULL
|
NULL
|
NULL
|
NULL
|
|
65086
|
2280
|
10
|
2026-05-20T15:10:06.907061+00:00
|
/Users/lukas/.screenpipe/data/data/2026-05-20/1779 /Users/lukas/.screenpipe/data/data/2026-05-20/1779289806907_m2.jpg...
|
Firefox
|
Inbox - kovaliklukas@gmail.com - Gmail — Personal
|
1
|
mail.google.com/mail/u/0/#inbox
|
monitor_2
|
NULL
|
NULL
|
NULL
|
NULL
|
Inbox - [EMAIL] - Gmail
Inbox - [EMAIL] - Gmail
Cl Inbox - [EMAIL] - Gmail
Inbox - [EMAIL] - Gmail
Close tab
DXP4800PLUS-B5F8
DXP4800PLUS-B5F8
(191) TED LASSO BEST SCENES - PART 2. The Most Heartwarming Ted Lasso Show Moments. - YouTube
(191) TED LASSO BEST SCENES - PART 2. The Most Heartwarming Ted Lasso Show Moments. - YouTube
LinkedIn Login, Sign in | LinkedIn
LinkedIn Login, Sign in | LinkedIn
Ambulatory sheet - Paperless-ngx
Ambulatory sheet - Paperless-ngx
New Tab
New Tab
💬1 - Apple MacBook Pro 14" Space Black M5 Max | Laptop.bg - Технологията с теб
💬1 - Apple MacBook Pro 14" Space Black M5 Max | Laptop.bg - Технологията с теб
Apple Mac Studio M4 Max | Desktop.bg - Мощни PC Гейминг конфигурации, Персонални компютри на изплащане за игри и работа
Apple Mac Studio M4 Max | Desktop.bg - Мощни PC Гейминг конфигурации, Персонални компютри на изплащане за игри и работа
Welcome to authentik! - authentik
Welcome to authentik! - authentik
Завеждане на щета онлайн | Euroins
Завеждане на щета онлайн | Euroins
New Tab
Customize sidebar
Open Google Gemini (⌃X)
Open history (⇧⌘H)
Open bookmarks (⌘B)
Bitwarden
None selected
Skip to content
Skip to content
Using Gmail with screen readers
Using Gmail with screen readers
Main menu
Gmail
Search
Search
Search mail
Advanced search options
Search mail
Support
Settings
Ask Gemini
Google apps
Google Account: Lukáš Koválik ([EMAIL])
Compose
Labels
Labels
Inbox
Inbox
Starred
Starred
Snoozed
Snoozed
Important
Important
Sent
Sent
Drafts 8 unread
Drafts
8
Purchases has menu
Purchases
Social 5224 unread has menu
Social
5,224
Updates 8834 unread has menu
Updates
8,834
Forums 6161 unread has menu
Forums
6,161
Promotions 38854 unread has menu
Promotions
38,854
More labels
More
Labels
Labels
Create new label
Labels
Labels
[Imap]/Nevyžiadaná pošta has menu
[Imap]/Nevyžiadaná pošta
arch has menu
arch
Deleted Items has menu
Deleted Items
Fibank 1229 unread has menu
Fibank
1,229
FL 6 unread has menu
FL
6
Hardware & Software has menu
Hardware & Software
HOSTING 5 unread has menu
HOSTING
5
Infected Items has menu
Infected Items
jiminny-github 7560 unread has menu
jiminny-github
7,560
Junk E-mail 219 unread has menu
Junk E-mail
219
Kontakty has menu
Kontakty
Sent Items has menu
Sent Items
WORK 848 unread has menu
WORK
848
z centra 1274 unread has menu
z centra
1,274
More labels
More
Select
Refresh
More email options
Show more messages
1
–
50
of
21,333
Newer
Older
Toggle split pane mode
Input tools on/off (Ctrl-Shift-K)
Select input tool
Conversations
Conversations
Primary
Primary
Primary
Promotions, 50 new messages,
Promotions, 50 new messages,
50 new
Promotions
Social, 18 new messages,
Social, 18 new messages,
18 new
Social
Updates
Updates
Updates
CircleCI Builds, [CircleCI] Workflow canceled: jiminny / app on JY-20676-delete-report-related-objects, 17:06, app / build_accept_deploy jiminny / app 2026-05-20T13:56:47Z JY-20676-delete-report-related-objects #58634 Commit push by Lukas Kovalik (LakyLak) 4f15440 JY-20676 fix tests Authored by Lukas Kovalik (.
Not starred
Important mainly because it was sent directly to you.
CircleCI Builds
[CircleCI] Workflow canceled: jiminny / app on JY-20676-delete-report-related-objects - app / build_accept_deploy jiminny / app 2026-05-20T13:56:47Z JY-20676-delete-report-related-objects #58634 Commit push by Lukas Kovalik (LakyLak) 4f15440 JY-20676 fix tests Authored by Lukas Kovalik (
[CircleCI] Workflow canceled: jiminny / app on JY-20676-delete-report-related-objects
-
app / build_accept_deploy jiminny / app 2026-05-20T13:56:47Z JY-20676-delete-report-related-objects #58634 Commit push by Lukas Kovalik (LakyLak) 4f15440 JY-20676 fix tests Authored by Lukas Kovalik (
17:06
CircleCI Builds, [CircleCI] Workflow failed: jiminny / app on JY-20676-delete-report-related-objects, 15:18, app / build_accept_deploy jiminny / app 2026-05-20T12:03:29Z JY-20676-delete-report-related-objects #58619 Commit push by Lukas Kovalik (LakyLak) 84a3b9c JY-20676 code review suggestions Authored by.
Not starred
Important mainly because it was sent directly to you.
CircleCI Builds
[CircleCI] Workflow failed: jiminny / app on JY-20676-delete-report-related-objects - app / build_accept_deploy jiminny / app 2026-05-20T12:03:29Z JY-20676-delete-report-related-objects #58619 Commit push by Lukas Kovalik (LakyLak) 84a3b9c JY-20676 code review suggestions Authored by
[CircleCI] Workflow failed: jiminny / app on JY-20676-delete-report-related-objects
-
app / build_accept_deploy jiminny / app 2026-05-20T12:03:29Z JY-20676-delete-report-related-objects #58619 Commit push by Lukas Kovalik (LakyLak) 84a3b9c JY-20676 code review suggestions Authored by
15:18
CircleCI Builds, [CircleCI] Workflow canceled: jiminny / app on JY-20613-allow-owner-role-on-team-setup, 14:44, app / build_accept_deploy jiminny / app 2026-05-20T11:44:29Z JY-20613-allow-owner-role-on-team-setup #58611 Commit push by Lukas Kovalik (LakyLak) 3642c93 JY-20613 remove recoder and voice role from.
Not starred
Important mainly because it was sent directly to you.
CircleCI Builds...
|
[{"role":"AXRadioButton","text [{"role":"AXRadioButton","text":"Inbox - kovaliklukas@gmail.com - Gmail","depth":4,"bounds":{"left":0.0,"top":0.0518755,"width":0.06881649,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"Inbox - kovaliklukas@gmail.com - Gmail","depth":5,"bounds":{"left":0.013297873,"top":0.06304868,"width":0.06898271,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Close tab","depth":5,"bounds":{"left":0.05651596,"top":0.05905826,"width":0.007978723,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXRadioButton","text":"DXP4800PLUS-B5F8","depth":4,"bounds":{"left":0.0,"top":0.08459697,"width":0.06881649,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"DXP4800PLUS-B5F8","depth":5,"bounds":{"left":0.013297873,"top":0.09577015,"width":0.036901597,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"(191) TED LASSO BEST SCENES - PART 2. The Most Heartwarming Ted Lasso Show Moments. - YouTube","depth":4,"bounds":{"left":0.0,"top":0.11731844,"width":0.06881649,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"(191) TED LASSO BEST SCENES - PART 2. The Most Heartwarming Ted Lasso Show Moments. - YouTube","depth":5,"bounds":{"left":0.013297873,"top":0.12849163,"width":0.18218085,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"LinkedIn Login, Sign in | LinkedIn","depth":4,"bounds":{"left":0.0,"top":0.15003991,"width":0.06881649,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LinkedIn Login, Sign in | LinkedIn","depth":5,"bounds":{"left":0.013297873,"top":0.16121309,"width":0.05668218,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Ambulatory sheet - Paperless-ngx","depth":4,"bounds":{"left":0.0,"top":0.18276137,"width":0.06881649,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Ambulatory sheet - Paperless-ngx","depth":5,"bounds":{"left":0.013297873,"top":0.19393456,"width":0.059507977,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"New Tab","depth":4,"bounds":{"left":0.0,"top":0.21548285,"width":0.06881649,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"New Tab","depth":5,"bounds":{"left":0.013297873,"top":0.22665602,"width":0.014960106,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"💬1 - Apple MacBook Pro 14\" Space Black M5 Max | Laptop.bg - Технологията с теб","depth":4,"bounds":{"left":0.0,"top":0.2482043,"width":0.06881649,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"💬1 - Apple MacBook Pro 14\" Space Black M5 Max | Laptop.bg - Технологията с теб","depth":5,"bounds":{"left":0.013297873,"top":0.25937748,"width":0.14727394,"height":0.011572227},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Apple Mac Studio M4 Max | Desktop.bg - Мощни PC Гейминг конфигурации, Персонални компютри на изплащане за игри и работа","depth":4,"bounds":{"left":0.0,"top":0.28092578,"width":0.06881649,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Apple Mac Studio M4 Max | Desktop.bg - Мощни PC Гейминг конфигурации, Персонални компютри на изплащане за игри и работа","depth":5,"bounds":{"left":0.013297873,"top":0.29209897,"width":0.2365359,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Welcome to authentik! - authentik","depth":4,"bounds":{"left":0.0,"top":0.31364724,"width":0.06881649,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Welcome to authentik! - authentik","depth":5,"bounds":{"left":0.013297873,"top":0.32482043,"width":0.05900931,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Завеждане на щета онлайн | Euroins","depth":4,"bounds":{"left":0.0,"top":0.3463687,"width":0.06881649,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Завеждане на щета онлайн | Euroins","depth":5,"bounds":{"left":0.013297873,"top":0.3575419,"width":0.0653258,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Tab","depth":4,"bounds":{"left":0.0028257978,"top":0.38068634,"width":0.06333112,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Customize sidebar","depth":6,"bounds":{"left":0.0028257978,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open Google Gemini (⌃X)","depth":6,"bounds":{"left":0.013796543,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open history (⇧⌘H)","depth":6,"bounds":{"left":0.024933511,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open bookmarks (⌘B)","depth":6,"bounds":{"left":0.036070477,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bitwarden","depth":6,"bounds":{"left":0.04720745,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"None selected","depth":8,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Skip to content","depth":10,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Skip to content","depth":11,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Using Gmail with screen readers","depth":10,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXStaticText","text":"Using Gmail with screen readers","depth":11,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Main menu","depth":11,"bounds":{"left":0.07280585,"top":0.058260176,"width":0.015957447,"height":0.03830806},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":true},{"role":"AXLink","text":"Gmail","depth":12,"bounds":{"left":0.09009308,"top":0.061452515,"width":0.036236703,"height":0.035115723},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Search","depth":12,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Search","depth":13,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXTextField","text":"Search mail","depth":18,"bounds":{"left":0.17220744,"top":0.06943336,"width":0.18916224,"height":0.016360734},"on_screen":true,"help_text":"","role_description":"text field","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Advanced search options","depth":12,"bounds":{"left":0.37466756,"top":0.058260176,"width":0.01861702,"height":0.03671189},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Search mail","depth":12,"bounds":{"left":0.15392287,"top":0.058260176,"width":0.01861702,"height":0.03671189},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":false,"is_focused":false,"is_selected":false},{"role":"AXMenuButton","text":"Support","depth":12,"bounds":{"left":0.66356385,"top":0.061452515,"width":0.013297873,"height":0.031923383},"on_screen":true,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXMenuButton","text":"Settings","depth":12,"bounds":{"left":0.6781915,"top":0.061452515,"width":0.013297873,"height":0.031923383},"on_screen":true,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Ask Gemini","depth":12,"bounds":{"left":0.6921542,"top":0.061452515,"width":0.013297873,"height":0.031923383},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Google apps","depth":14,"bounds":{"left":0.70611703,"top":0.061452515,"width":0.013297873,"height":0.031923383},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Google Account: Lukáš Koválik (kovaliklukas@gmail.com)","depth":14,"bounds":{"left":0.72207445,"top":0.061452515,"width":0.013297873,"height":0.031923383},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Compose","depth":9,"bounds":{"left":0.071476065,"top":0.10933759,"width":0.04737367,"height":0.031923383},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Labels","depth":13,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Labels","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Inbox","depth":17,"bounds":{"left":0.09009308,"top":0.15003991,"width":0.012466756,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Inbox","depth":18,"bounds":{"left":0.09009308,"top":0.15003991,"width":0.012466756,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Starred","depth":17,"bounds":{"left":0.09009308,"top":0.16919394,"width":0.015625,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Starred","depth":18,"bounds":{"left":0.09009308,"top":0.16919394,"width":0.015625,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Snoozed","depth":17,"bounds":{"left":0.09009308,"top":0.18834797,"width":0.018284574,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Snoozed","depth":18,"bounds":{"left":0.09009308,"top":0.18834797,"width":0.018284574,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Important","depth":18,"bounds":{"left":0.09009308,"top":0.207502,"width":0.020777926,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Important","depth":19,"bounds":{"left":0.09009308,"top":0.207502,"width":0.020777926,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Sent","depth":17,"bounds":{"left":0.09009308,"top":0.22665602,"width":0.009640957,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Sent","depth":18,"bounds":{"left":0.09009308,"top":0.22665602,"width":0.009640957,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Drafts 8 unread","depth":17,"bounds":{"left":0.09009308,"top":0.24581006,"width":0.013796543,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Drafts","depth":18,"bounds":{"left":0.09009308,"top":0.24581006,"width":0.013796543,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"8","depth":17,"bounds":{"left":0.14378324,"top":0.24700718,"width":0.0021609042,"height":0.012370312},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Purchases has menu","depth":17,"bounds":{"left":0.09009308,"top":0.26496407,"width":0.021941489,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Purchases","depth":18,"bounds":{"left":0.09009308,"top":0.26496407,"width":0.021941489,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Social 5224 unread has menu","depth":17,"bounds":{"left":0.09009308,"top":0.28411812,"width":0.013796543,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Social","depth":18,"bounds":{"left":0.09009308,"top":0.28411812,"width":0.013796543,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5,224","depth":17,"bounds":{"left":0.13613696,"top":0.28531525,"width":0.009807181,"height":0.012370312},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Updates 8834 unread has menu","depth":17,"bounds":{"left":0.09009308,"top":0.30327216,"width":0.018949468,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Updates","depth":18,"bounds":{"left":0.09009308,"top":0.30327216,"width":0.018949468,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"8,834","depth":17,"bounds":{"left":0.13613696,"top":0.3044693,"width":0.009807181,"height":0.012370312},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Forums 6161 unread has menu","depth":17,"bounds":{"left":0.09009308,"top":0.32242617,"width":0.016788565,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Forums","depth":18,"bounds":{"left":0.09009308,"top":0.32242617,"width":0.016788565,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"6,161","depth":17,"bounds":{"left":0.13746676,"top":0.3236233,"width":0.008477394,"height":0.012370312},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Promotions 38854 unread has menu","depth":17,"bounds":{"left":0.09009308,"top":0.3415802,"width":0.025930852,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Promotions","depth":18,"bounds":{"left":0.09009308,"top":0.3415802,"width":0.025930852,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"38,854","depth":17,"bounds":{"left":0.13397606,"top":0.34277734,"width":0.011968086,"height":0.012370312},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"More labels","depth":13,"bounds":{"left":0.06881649,"top":0.35834,"width":0.07978723,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"More","depth":15,"bounds":{"left":0.09009308,"top":0.36073422,"width":0.010804521,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Labels","depth":12,"bounds":{"left":0.0774601,"top":0.39984038,"width":0.061835106,"height":0.016360734},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Labels","depth":13,"bounds":{"left":0.0774601,"top":0.39984038,"width":0.016456118,"height":0.016360734},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Create new label","depth":12,"bounds":{"left":0.1392952,"top":0.40023944,"width":0.0066489363,"height":0.015961692},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Labels","depth":13,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Labels","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"[Imap]/Nevyžiadaná pošta has menu","depth":17,"bounds":{"left":0.09009308,"top":0.42817238,"width":0.055352394,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"[Imap]/Nevyžiadaná pošta","depth":18,"bounds":{"left":0.09009308,"top":0.42817238,"width":0.055352394,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"arch has menu","depth":17,"bounds":{"left":0.09009308,"top":0.44732642,"width":0.009474734,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"arch","depth":18,"bounds":{"left":0.09009308,"top":0.44732642,"width":0.009474734,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Deleted Items has menu","depth":17,"bounds":{"left":0.09009308,"top":0.46648043,"width":0.029089095,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Deleted Items","depth":18,"bounds":{"left":0.09009308,"top":0.46648043,"width":0.029089095,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Fibank 1229 unread has menu","depth":17,"bounds":{"left":0.09009308,"top":0.48563448,"width":0.01512633,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Fibank","depth":18,"bounds":{"left":0.09009308,"top":0.48563448,"width":0.01512633,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"1,229","depth":17,"bounds":{"left":0.13680187,"top":0.4868316,"width":0.009142287,"height":0.012370312},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"FL 6 unread has menu","depth":17,"bounds":{"left":0.09009308,"top":0.5047885,"width":0.005319149,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"FL","depth":18,"bounds":{"left":0.09009308,"top":0.5047885,"width":0.005319149,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"6","depth":17,"bounds":{"left":0.14361702,"top":0.5059856,"width":0.0023271276,"height":0.012370312},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Hardware & Software has menu","depth":17,"bounds":{"left":0.09009308,"top":0.52394253,"width":0.044714097,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Hardware & Software","depth":18,"bounds":{"left":0.09009308,"top":0.52394253,"width":0.044714097,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"HOSTING 5 unread has menu","depth":17,"bounds":{"left":0.09009308,"top":0.54309654,"width":0.02144282,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"HOSTING","depth":18,"bounds":{"left":0.09009308,"top":0.54309654,"width":0.02144282,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5","depth":17,"bounds":{"left":0.14378324,"top":0.5442937,"width":0.0021609042,"height":0.012370312},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Infected Items has menu","depth":17,"bounds":{"left":0.09009308,"top":0.5622506,"width":0.030086435,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Infected Items","depth":18,"bounds":{"left":0.09009308,"top":0.5622506,"width":0.030086435,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"jiminny-github 7560 unread has menu","depth":18,"bounds":{"left":0.09009308,"top":0.5814046,"width":0.03324468,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"jiminny-github","depth":19,"bounds":{"left":0.09009308,"top":0.5814046,"width":0.03324468,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"7,560","depth":18,"bounds":{"left":0.13613696,"top":0.5826017,"width":0.009807181,"height":0.012370312},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Junk E-mail 219 unread has menu","depth":17,"bounds":{"left":0.09009308,"top":0.60055864,"width":0.026761968,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Junk E-mail","depth":18,"bounds":{"left":0.09009308,"top":0.60055864,"width":0.026761968,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"219","depth":17,"bounds":{"left":0.13996011,"top":0.6017558,"width":0.005984043,"height":0.012370312},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Kontakty has menu","depth":17,"bounds":{"left":0.09009308,"top":0.6197127,"width":0.018450798,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Kontakty","depth":18,"bounds":{"left":0.09009308,"top":0.6197127,"width":0.018450798,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Sent Items has menu","depth":17,"bounds":{"left":0.09009308,"top":0.6388667,"width":0.022273935,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Sent Items","depth":18,"bounds":{"left":0.09009308,"top":0.6388667,"width":0.022273935,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"WORK 848 unread has menu","depth":17,"bounds":{"left":0.09009308,"top":0.65802073,"width":0.014461436,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"WORK","depth":18,"bounds":{"left":0.09009308,"top":0.65802073,"width":0.014461436,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"848","depth":17,"bounds":{"left":0.1392952,"top":0.6592179,"width":0.0066489363,"height":0.012370312},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"z centra 1274 unread has menu","depth":17,"bounds":{"left":0.09009308,"top":0.6771748,"width":0.018118352,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"z centra","depth":18,"bounds":{"left":0.09009308,"top":0.6771748,"width":0.018118352,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"1,274","depth":17,"bounds":{"left":0.13713431,"top":0.6783719,"width":0.00880984,"height":0.012370312},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"More labels","depth":13,"bounds":{"left":0.06881649,"top":0.69393456,"width":0.07978723,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"More","depth":15,"bounds":{"left":0.09009308,"top":0.6963288,"width":0.010804521,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXMenuButton","text":"Select","depth":12,"bounds":{"left":0.15924202,"top":0.11412609,"width":0.013297873,"height":0.015961692},"on_screen":true,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Refresh","depth":12,"bounds":{"left":0.17918883,"top":0.11412609,"width":0.0066489363,"height":0.015961692},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"More email options","depth":12,"bounds":{"left":0.1924867,"top":0.11412609,"width":0.0066489363,"height":0.015961692},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Show more messages","depth":13,"bounds":{"left":0.61702126,"top":0.10774142,"width":0.03125,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXStaticText","text":"1","depth":15,"bounds":{"left":0.6196808,"top":0.11612131,"width":0.0016622341,"height":0.012370312},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"–","depth":15,"bounds":{"left":0.6213431,"top":0.11612131,"width":0.002493351,"height":0.012370312},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"50","depth":15,"bounds":{"left":0.62383646,"top":0.11612131,"width":0.0048204786,"height":0.012370312},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"of","depth":15,"bounds":{"left":0.6286569,"top":0.11612131,"width":0.0056515955,"height":0.012370312},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"21,333","depth":15,"bounds":{"left":0.6343085,"top":0.11612131,"width":0.011303191,"height":0.012370312},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Newer","depth":13,"bounds":{"left":0.65226066,"top":0.11412609,"width":0.0066489363,"height":0.015961692},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":false,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Older","depth":13,"bounds":{"left":0.6655585,"top":0.11412609,"width":0.0066489363,"height":0.015961692},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Toggle split pane mode","depth":12,"bounds":{"left":0.6788564,"top":0.11412609,"width":0.0066489363,"height":0.015961692},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Input tools on/off (Ctrl-Shift-K)","depth":14,"bounds":{"left":0.6974734,"top":0.11412609,"width":0.0066489363,"height":0.015961692},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXMenuButton","text":"Select input tool","depth":14,"bounds":{"left":0.70412236,"top":0.11412609,"width":0.0066489363,"height":0.015961692},"on_screen":true,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Conversations","depth":13,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Conversations","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Primary","depth":15,"bounds":{"left":0.15392287,"top":0.14126097,"width":0.084109046,"height":0.03830806},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Primary","depth":16,"bounds":{"left":0.15392287,"top":0.14126097,"width":0.08344415,"height":0.03830806},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"Primary","depth":19,"bounds":{"left":0.17121011,"top":0.1528332,"width":0.016954787,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Promotions, 50 new messages,","depth":15,"bounds":{"left":0.23803191,"top":0.14126097,"width":0.084109046,"height":0.03830806},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Promotions, 50 new messages,","depth":16,"bounds":{"left":0.23803191,"top":0.14126097,"width":0.08377659,"height":0.03830806},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"50 new","depth":19,"bounds":{"left":0.28424203,"top":0.15403032,"width":0.013630319,"height":0.012370312},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Promotions","depth":19,"bounds":{"left":0.25531915,"top":0.1528332,"width":0.024933511,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Social, 18 new messages,","depth":15,"bounds":{"left":0.32214096,"top":0.14126097,"width":0.084109046,"height":0.03830806},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Social, 18 new messages,","depth":16,"bounds":{"left":0.32214096,"top":0.14126097,"width":0.08377659,"height":0.03830806},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"18 new","depth":19,"bounds":{"left":0.3565492,"top":0.15403032,"width":0.012799202,"height":0.012370312},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Social","depth":19,"bounds":{"left":0.3394282,"top":0.1528332,"width":0.013131649,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Updates","depth":15,"bounds":{"left":0.40625,"top":0.14126097,"width":0.084109046,"height":0.03830806},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Updates","depth":16,"bounds":{"left":0.40625,"top":0.14126097,"width":0.08377659,"height":0.03830806},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Updates","depth":19,"bounds":{"left":0.42353722,"top":0.1528332,"width":0.018284574,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"CircleCI Builds, [CircleCI] Workflow canceled: jiminny / app on JY-20676-delete-report-related-objects, 17:06, app / build_accept_deploy jiminny / app 2026-05-20T13:56:47Z JY-20676-delete-report-related-objects #58634 Commit push by Lukas Kovalik (LakyLak) 4f15440 JY-20676 fix tests Authored by Lukas Kovalik (.","depth":18,"bounds":{"left":0.15924202,"top":0.18276137,"width":0.0066489363,"height":0.015961692},"on_screen":true,"help_text":"","role_description":"checkbox","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Not starred","depth":18,"bounds":{"left":0.16921543,"top":0.18276137,"width":0.0066489363,"height":0.015961692},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Important mainly because it was sent directly to you.","depth":18,"bounds":{"left":0.17918883,"top":0.18276137,"width":0.0066489363,"height":0.015961692},"on_screen":true,"help_text":"","role_description":"switch","subrole":"AXSwitch","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"CircleCI Builds","depth":20,"bounds":{"left":0.18916224,"top":0.18355946,"width":0.030418882,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"[CircleCI] Workflow canceled: jiminny / app on JY-20676-delete-report-related-objects - app / build_accept_deploy jiminny / app 2026-05-20T13:56:47Z JY-20676-delete-report-related-objects #58634 Commit push by Lukas Kovalik (LakyLak) 4f15440 JY-20676 fix tests Authored by Lukas Kovalik (","depth":18,"bounds":{"left":0.2556516,"top":0.18276137,"width":0.42918882,"height":0.015961692},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"[CircleCI] Workflow canceled: jiminny / app on JY-20676-delete-report-related-objects","depth":22,"bounds":{"left":0.2556516,"top":0.18355946,"width":0.18301196,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"-","depth":22,"bounds":{"left":0.43866357,"top":0.18355946,"width":0.004488032,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"app / build_accept_deploy jiminny / app 2026-05-20T13:56:47Z JY-20676-delete-report-related-objects #58634 Commit push by Lukas Kovalik (LakyLak) 4f15440 JY-20676 fix tests Authored by Lukas Kovalik (","depth":21,"bounds":{"left":0.4431516,"top":0.18355946,"width":0.43916222,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"17:06","depth":19,"bounds":{"left":0.70644945,"top":0.18475658,"width":0.009640957,"height":0.012370312},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"CircleCI Builds, [CircleCI] Workflow failed: jiminny / app on JY-20676-delete-report-related-objects, 15:18, app / build_accept_deploy jiminny / app 2026-05-20T12:03:29Z JY-20676-delete-report-related-objects #58619 Commit push by Lukas Kovalik (LakyLak) 84a3b9c JY-20676 code review suggestions Authored by.","depth":18,"bounds":{"left":0.15924202,"top":0.20510775,"width":0.0066489363,"height":0.015961692},"on_screen":true,"help_text":"","role_description":"checkbox","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Not starred","depth":18,"bounds":{"left":0.16921543,"top":0.20510775,"width":0.0066489363,"height":0.015961692},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Important mainly because it was sent directly to you.","depth":18,"bounds":{"left":0.17918883,"top":0.20510775,"width":0.0066489363,"height":0.015961692},"on_screen":true,"help_text":"","role_description":"switch","subrole":"AXSwitch","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"CircleCI Builds","depth":20,"bounds":{"left":0.18916224,"top":0.20590582,"width":0.030418882,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"[CircleCI] Workflow failed: jiminny / app on JY-20676-delete-report-related-objects - app / build_accept_deploy jiminny / app 2026-05-20T12:03:29Z JY-20676-delete-report-related-objects #58619 Commit push by Lukas Kovalik (LakyLak) 84a3b9c JY-20676 code review suggestions Authored by","depth":18,"bounds":{"left":0.2556516,"top":0.20510775,"width":0.42918882,"height":0.015961692},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"[CircleCI] Workflow failed: jiminny / app on JY-20676-delete-report-related-objects","depth":22,"bounds":{"left":0.2556516,"top":0.20590582,"width":0.17553191,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"-","depth":22,"bounds":{"left":0.43118352,"top":0.20590582,"width":0.004488032,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"app / build_accept_deploy jiminny / app 2026-05-20T12:03:29Z JY-20676-delete-report-related-objects #58619 Commit push by Lukas Kovalik (LakyLak) 84a3b9c JY-20676 code review suggestions Authored by","depth":21,"bounds":{"left":0.43567154,"top":0.20590582,"width":0.4431516,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"15:18","depth":19,"bounds":{"left":0.7072806,"top":0.20710295,"width":0.00880984,"height":0.012370312},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"CircleCI Builds, [CircleCI] Workflow canceled: jiminny / app on JY-20613-allow-owner-role-on-team-setup, 14:44, app / build_accept_deploy jiminny / app 2026-05-20T11:44:29Z JY-20613-allow-owner-role-on-team-setup #58611 Commit push by Lukas Kovalik (LakyLak) 3642c93 JY-20613 remove recoder and voice role from.","depth":18,"bounds":{"left":0.15924202,"top":0.22745411,"width":0.0066489363,"height":0.015961692},"on_screen":true,"help_text":"","role_description":"checkbox","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Not starred","depth":18,"bounds":{"left":0.16921543,"top":0.22745411,"width":0.0066489363,"height":0.015961692},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Important mainly because it was sent directly to you.","depth":18,"bounds":{"left":0.17918883,"top":0.22745411,"width":0.0066489363,"height":0.015961692},"on_screen":true,"help_text":"","role_description":"switch","subrole":"AXSwitch","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"CircleCI Builds","depth":20,"bounds":{"left":0.18916224,"top":0.22825219,"width":0.030418882,"height":0.014764565},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"}]...
|
3645857074399183574
|
6698002377854652141
|
visual_change
|
accessibility
|
NULL
|
Inbox - [EMAIL] - Gmail
Inbox - [EMAIL] - Gmail
Cl Inbox - [EMAIL] - Gmail
Inbox - [EMAIL] - Gmail
Close tab
DXP4800PLUS-B5F8
DXP4800PLUS-B5F8
(191) TED LASSO BEST SCENES - PART 2. The Most Heartwarming Ted Lasso Show Moments. - YouTube
(191) TED LASSO BEST SCENES - PART 2. The Most Heartwarming Ted Lasso Show Moments. - YouTube
LinkedIn Login, Sign in | LinkedIn
LinkedIn Login, Sign in | LinkedIn
Ambulatory sheet - Paperless-ngx
Ambulatory sheet - Paperless-ngx
New Tab
New Tab
💬1 - Apple MacBook Pro 14" Space Black M5 Max | Laptop.bg - Технологията с теб
💬1 - Apple MacBook Pro 14" Space Black M5 Max | Laptop.bg - Технологията с теб
Apple Mac Studio M4 Max | Desktop.bg - Мощни PC Гейминг конфигурации, Персонални компютри на изплащане за игри и работа
Apple Mac Studio M4 Max | Desktop.bg - Мощни PC Гейминг конфигурации, Персонални компютри на изплащане за игри и работа
Welcome to authentik! - authentik
Welcome to authentik! - authentik
Завеждане на щета онлайн | Euroins
Завеждане на щета онлайн | Euroins
New Tab
Customize sidebar
Open Google Gemini (⌃X)
Open history (⇧⌘H)
Open bookmarks (⌘B)
Bitwarden
None selected
Skip to content
Skip to content
Using Gmail with screen readers
Using Gmail with screen readers
Main menu
Gmail
Search
Search
Search mail
Advanced search options
Search mail
Support
Settings
Ask Gemini
Google apps
Google Account: Lukáš Koválik ([EMAIL])
Compose
Labels
Labels
Inbox
Inbox
Starred
Starred
Snoozed
Snoozed
Important
Important
Sent
Sent
Drafts 8 unread
Drafts
8
Purchases has menu
Purchases
Social 5224 unread has menu
Social
5,224
Updates 8834 unread has menu
Updates
8,834
Forums 6161 unread has menu
Forums
6,161
Promotions 38854 unread has menu
Promotions
38,854
More labels
More
Labels
Labels
Create new label
Labels
Labels
[Imap]/Nevyžiadaná pošta has menu
[Imap]/Nevyžiadaná pošta
arch has menu
arch
Deleted Items has menu
Deleted Items
Fibank 1229 unread has menu
Fibank
1,229
FL 6 unread has menu
FL
6
Hardware & Software has menu
Hardware & Software
HOSTING 5 unread has menu
HOSTING
5
Infected Items has menu
Infected Items
jiminny-github 7560 unread has menu
jiminny-github
7,560
Junk E-mail 219 unread has menu
Junk E-mail
219
Kontakty has menu
Kontakty
Sent Items has menu
Sent Items
WORK 848 unread has menu
WORK
848
z centra 1274 unread has menu
z centra
1,274
More labels
More
Select
Refresh
More email options
Show more messages
1
–
50
of
21,333
Newer
Older
Toggle split pane mode
Input tools on/off (Ctrl-Shift-K)
Select input tool
Conversations
Conversations
Primary
Primary
Primary
Promotions, 50 new messages,
Promotions, 50 new messages,
50 new
Promotions
Social, 18 new messages,
Social, 18 new messages,
18 new
Social
Updates
Updates
Updates
CircleCI Builds, [CircleCI] Workflow canceled: jiminny / app on JY-20676-delete-report-related-objects, 17:06, app / build_accept_deploy jiminny / app 2026-05-20T13:56:47Z JY-20676-delete-report-related-objects #58634 Commit push by Lukas Kovalik (LakyLak) 4f15440 JY-20676 fix tests Authored by Lukas Kovalik (.
Not starred
Important mainly because it was sent directly to you.
CircleCI Builds
[CircleCI] Workflow canceled: jiminny / app on JY-20676-delete-report-related-objects - app / build_accept_deploy jiminny / app 2026-05-20T13:56:47Z JY-20676-delete-report-related-objects #58634 Commit push by Lukas Kovalik (LakyLak) 4f15440 JY-20676 fix tests Authored by Lukas Kovalik (
[CircleCI] Workflow canceled: jiminny / app on JY-20676-delete-report-related-objects
-
app / build_accept_deploy jiminny / app 2026-05-20T13:56:47Z JY-20676-delete-report-related-objects #58634 Commit push by Lukas Kovalik (LakyLak) 4f15440 JY-20676 fix tests Authored by Lukas Kovalik (
17:06
CircleCI Builds, [CircleCI] Workflow failed: jiminny / app on JY-20676-delete-report-related-objects, 15:18, app / build_accept_deploy jiminny / app 2026-05-20T12:03:29Z JY-20676-delete-report-related-objects #58619 Commit push by Lukas Kovalik (LakyLak) 84a3b9c JY-20676 code review suggestions Authored by.
Not starred
Important mainly because it was sent directly to you.
CircleCI Builds
[CircleCI] Workflow failed: jiminny / app on JY-20676-delete-report-related-objects - app / build_accept_deploy jiminny / app 2026-05-20T12:03:29Z JY-20676-delete-report-related-objects #58619 Commit push by Lukas Kovalik (LakyLak) 84a3b9c JY-20676 code review suggestions Authored by
[CircleCI] Workflow failed: jiminny / app on JY-20676-delete-report-related-objects
-
app / build_accept_deploy jiminny / app 2026-05-20T12:03:29Z JY-20676-delete-report-related-objects #58619 Commit push by Lukas Kovalik (LakyLak) 84a3b9c JY-20676 code review suggestions Authored by
15:18
CircleCI Builds, [CircleCI] Workflow canceled: jiminny / app on JY-20613-allow-owner-role-on-team-setup, 14:44, app / build_accept_deploy jiminny / app 2026-05-20T11:44:29Z JY-20613-allow-owner-role-on-team-setup #58611 Commit push by Lukas Kovalik (LakyLak) 3642c93 JY-20613 remove recoder and voice role from.
Not starred
Important mainly because it was sent directly to you.
CircleCI Builds...
|
65085
|
NULL
|
NULL
|
NULL
|
|
65087
|
2279
|
9
|
2026-05-20T15:10:08.302261+00:00
|
/Users/lukas/.screenpipe/data/data/2026-05-20/1779 /Users/lukas/.screenpipe/data/data/2026-05-20/1779289808302_m1.jpg...
|
Firefox
|
Inbox - kovaliklukas@gmail.com - Gmail — Personal
|
1
|
mail.google.com/mail/u/0/#inbox
|
monitor_1
|
NULL
|
NULL
|
NULL
|
NULL
|
SlackFileEditViewGoHistoryWindowHelp> 0 lll2010 SlackFileEditViewGoHistoryWindowHelp> 0 lll20100% C4 8• Wed 20 May 18:10:07APP (-zsh)|181DOCKER• ₴1DEV (-zsh)₴2screenpipe"283APP (-zsh)Fixed 0 of 5690 files in 78.144 seconds, 60.00MBmemoryusedWhat's next:Try Docker Debug for seamless, persistentdebugging tools in any containeror image → docker debug docker_lamp_1Learn more at [URL_WITH_CREDENTIALS] (JY-20676-delete-report-related-objects) $ sips -s format png ~/Downloads/Photos-3-001\(2\)/talon-f.HEIC --out ~/Downloads/Photos-3-001\(2\)/talon-f.png/Users/lukas/Downloads/Photos-3-001(2)/talon-f.HEIC/Users/lukas/Downloads/Photos-3-001(2)/talon-f.pnglukas@Lukas-Kovaliks-MacBook-Pro-Jiminny ~/jiminny/app(JY-20676-delete-report-related-objects) $ sips -s format png ~/Downloads/Photos-3-001\(2\)/talon-b.HEIC --out ~/Downloads/Photos-3-001\(2\)/talon-b.png/Users/lukas/Downloads/Photos-3-001(2)/talon-b.HEIC/Users/lukas/Downloads/Photos-3-001(2)/talon-b.pnglukas@Lukas-Kovaliks-MacBook-Pro-Jiminny ~/jiminny/app(JY-20676-delete-report-related-objects) $ sips -s format png ~/Downloads/Photos-3-001\(2\)/vin.HEIC --out ~/Downloads/Photos-3-001\(2\)/vin.png/Users/lukas/Downloads/Photos-3-001(2)/vin.HEIC/Users/lukas/Downloads/Photos-3-001(2)/vin.pnglukas@Lukas-Kovaliks-MacBook-Pro-Jiminny ~/jiminny/app (JY-20676-delete-report-related-objects) $ sips -s format png ~/Downloads/Photos-3-001\(2\)/dr-lic-f.HEIC --out ~/Downloads/Photos-3-001\(2\)/dr-lic-f.png/Users/lukas/Downloads/Photos-3-001(2)/dr-lic-f.HEIC/Users/lukas/Downloads/Photos-3-001(2)/dr-lic-f.pnglukas@Lukas-Kovaliks-MacBook-Pro-Jiminny ~/jiminny/app (JY-20676-delete-report-related-objects) $ sips -s format png ~/Downloads/Photos-3-001\(2\)/dr-lic-b.HEIC --out ~/Downloads/Photos-3-001\(2\)/dr-lic-b.png/Users/lukas/Downloads/Photos-3-001(2)/dr-lic-b.HEIC/Users/lukas/Downloads/Photos-3-001(2)/dr-lic-b.pnglukas@Lukas-Kovaliks-MacBook-Pro-Jiminny ~/jiminny/app (JY-20676-delete-report-related-objects)sips -s format png ~/Downloads/Photos-3-001\(2\)/gtp.HEIC --out ~/Downloads/Photos-3-001\(2)/gtp.pngWarning: /Users/lukas/Downloads/Photos-3-001(2)/gtp.HEIC not a valid file - skippinglukas@Lukas-Kovaliks-MacBook-Pro-Jiminny ~/jiminny/app (JY-20676-delete-report-related-objects) $ sips -s format png ~/Downloads/Photos-3-001\(2\)/gtp.heif --out ~/Downloads/Photos-3-001\(2\)/gtp.png/Users/lukas/Downloads/Photos-3-001(2)/gtp.heif/Users/lukas/Downloads/Photos-3-001(2)/gtp.pnglukas@Lukas-Kovaliks-MacBook-Pro-Jiminny ~/jiminny/app (JY-20676-delete-report-related-objects) $ sips -s format png ~/Downloads/Photos-3-001\(2\)/talon-m-f.HEIC --out ~/Downloads/Photos-3-001\(2\)/talon-m-f.png/Users/lukas/Downloads/Photos-3-001(2)/talon-m-f.HEIC/Users/lukas/Downloads/Photos-3-001(2)/talon-m-f.pnglukas@Lukas-Kovaliks-MacBook-Pro-Jiminny ~/Jiminny/app(JY-20676-delete-report-related-objects) $ sips -s format png ~/Downloads/Photos-3-001\(2\)/talon-m-b.HEIC --out ~/Downloads/Photos-3-001\(2\)/talon-m-b.png/Users/lukas/Downloads/Photos-3-001(2)/talon-m-b.HEIC/Users/lukas/Downloads/Photos-3-001(2)/talon-m-b.pnglukas@Lukas-Kovaliks-MacBook-Pro-Jiminny ~/jiminny/app (JY-20676-delete-report-related-objects) $D...
|
NULL
|
-7257147370534093884
|
NULL
|
visual_change
|
ocr
|
NULL
|
SlackFileEditViewGoHistoryWindowHelp> 0 lll2010 SlackFileEditViewGoHistoryWindowHelp> 0 lll20100% C4 8• Wed 20 May 18:10:07APP (-zsh)|181DOCKER• ₴1DEV (-zsh)₴2screenpipe"283APP (-zsh)Fixed 0 of 5690 files in 78.144 seconds, 60.00MBmemoryusedWhat's next:Try Docker Debug for seamless, persistentdebugging tools in any containeror image → docker debug docker_lamp_1Learn more at [URL_WITH_CREDENTIALS] (JY-20676-delete-report-related-objects) $ sips -s format png ~/Downloads/Photos-3-001\(2\)/talon-f.HEIC --out ~/Downloads/Photos-3-001\(2\)/talon-f.png/Users/lukas/Downloads/Photos-3-001(2)/talon-f.HEIC/Users/lukas/Downloads/Photos-3-001(2)/talon-f.pnglukas@Lukas-Kovaliks-MacBook-Pro-Jiminny ~/jiminny/app(JY-20676-delete-report-related-objects) $ sips -s format png ~/Downloads/Photos-3-001\(2\)/talon-b.HEIC --out ~/Downloads/Photos-3-001\(2\)/talon-b.png/Users/lukas/Downloads/Photos-3-001(2)/talon-b.HEIC/Users/lukas/Downloads/Photos-3-001(2)/talon-b.pnglukas@Lukas-Kovaliks-MacBook-Pro-Jiminny ~/jiminny/app(JY-20676-delete-report-related-objects) $ sips -s format png ~/Downloads/Photos-3-001\(2\)/vin.HEIC --out ~/Downloads/Photos-3-001\(2\)/vin.png/Users/lukas/Downloads/Photos-3-001(2)/vin.HEIC/Users/lukas/Downloads/Photos-3-001(2)/vin.pnglukas@Lukas-Kovaliks-MacBook-Pro-Jiminny ~/jiminny/app (JY-20676-delete-report-related-objects) $ sips -s format png ~/Downloads/Photos-3-001\(2\)/dr-lic-f.HEIC --out ~/Downloads/Photos-3-001\(2\)/dr-lic-f.png/Users/lukas/Downloads/Photos-3-001(2)/dr-lic-f.HEIC/Users/lukas/Downloads/Photos-3-001(2)/dr-lic-f.pnglukas@Lukas-Kovaliks-MacBook-Pro-Jiminny ~/jiminny/app (JY-20676-delete-report-related-objects) $ sips -s format png ~/Downloads/Photos-3-001\(2\)/dr-lic-b.HEIC --out ~/Downloads/Photos-3-001\(2\)/dr-lic-b.png/Users/lukas/Downloads/Photos-3-001(2)/dr-lic-b.HEIC/Users/lukas/Downloads/Photos-3-001(2)/dr-lic-b.pnglukas@Lukas-Kovaliks-MacBook-Pro-Jiminny ~/jiminny/app (JY-20676-delete-report-related-objects)sips -s format png ~/Downloads/Photos-3-001\(2\)/gtp.HEIC --out ~/Downloads/Photos-3-001\(2)/gtp.pngWarning: /Users/lukas/Downloads/Photos-3-001(2)/gtp.HEIC not a valid file - skippinglukas@Lukas-Kovaliks-MacBook-Pro-Jiminny ~/jiminny/app (JY-20676-delete-report-related-objects) $ sips -s format png ~/Downloads/Photos-3-001\(2\)/gtp.heif --out ~/Downloads/Photos-3-001\(2\)/gtp.png/Users/lukas/Downloads/Photos-3-001(2)/gtp.heif/Users/lukas/Downloads/Photos-3-001(2)/gtp.pnglukas@Lukas-Kovaliks-MacBook-Pro-Jiminny ~/jiminny/app (JY-20676-delete-report-related-objects) $ sips -s format png ~/Downloads/Photos-3-001\(2\)/talon-m-f.HEIC --out ~/Downloads/Photos-3-001\(2\)/talon-m-f.png/Users/lukas/Downloads/Photos-3-001(2)/talon-m-f.HEIC/Users/lukas/Downloads/Photos-3-001(2)/talon-m-f.pnglukas@Lukas-Kovaliks-MacBook-Pro-Jiminny ~/Jiminny/app(JY-20676-delete-report-related-objects) $ sips -s format png ~/Downloads/Photos-3-001\(2\)/talon-m-b.HEIC --out ~/Downloads/Photos-3-001\(2\)/talon-m-b.png/Users/lukas/Downloads/Photos-3-001(2)/talon-m-b.HEIC/Users/lukas/Downloads/Photos-3-001(2)/talon-m-b.pnglukas@Lukas-Kovaliks-MacBook-Pro-Jiminny ~/jiminny/app (JY-20676-delete-report-related-objects) $D...
|
NULL
|
NULL
|
NULL
|
NULL
|
|
65221
|
2285
|
16
|
2026-05-20T15:29:32.955929+00:00
|
/Users/lukas/.screenpipe/data/data/2026-05-20/1779 /Users/lukas/.screenpipe/data/data/2026-05-20/1779290972955_m1.jpg...
|
Firefox
|
Inbox - kovaliklukas@gmail.com - Gmail — Personal
|
1
|
mail.google.com/mail/u/0/#inbox
|
monitor_1
|
NULL
|
NULL
|
NULL
|
NULL
|
SlackFileEditViewGoHistoryWindowHelp> 0 lllDOCK SlackFileEditViewGoHistoryWindowHelp> 0 lllDOCKERO ₴1DEV (-zsh)APP (-zsh)|₴2screenpipe"20О 83A100% C4 8• Wed 20 May 18:29:32181APP (-zsh)Fixed 0 of 5690 files in 78.144 seconds, 60.00MBmemoryusedWhat's next:Try Docker Debug for seamless, persistentdebugging tools in any containeror image → docker debug docker_lamp_1Learn more at [URL_WITH_CREDENTIALS] (JY-20676-delete-report-related-objects) $ sips -s format png ~/Downloads/Photos-3-001\(2\)/talon-f.HEIC --out ~/Downloads/Photos-3-001\(2\)/talon-f.png/Users/lukas/Downloads/Photos-3-001(2)/talon-f.HEIC/Users/lukas/Downloads/Photos-3-001(2)/talon-f.pnglukas@Lukas-Kovaliks-MacBook-Pro-Jiminny ~/jiminny/app(JY-20676-delete-report-related-objects) $ sips -s format png ~/Downloads/Photos-3-001\(2\)/talon-b.HEIC --out ~/Downloads/Photos-3-001\(2\)/talon-b.png/Users/lukas/Downloads/Photos-3-001(2)/talon-b.HEIC/Users/lukas/Downloads/Photos-3-001(2)/talon-b.pnglukas@Lukas-Kovaliks-MacBook-Pro-Jiminny ~/jiminny/app(JY-20676-delete-report-related-objects) $ sips -s format png ~/Downloads/Photos-3-001\(2\)/vin.HEIC --out ~/Downloads/Photos-3-001\(2\)/vin.png/Users/lukas/Downloads/Photos-3-001(2)/vin.HEIC/Users/lukas/Downloads/Photos-3-001(2)/vin.pnglukas@Lukas-Kovaliks-MacBook-Pro-Jiminny ~/jiminny/app (JY-20676-delete-report-related-objects) $ sips -s format png ~/Downloads/Photos-3-001\(2\)/dr-lic-f.HEIC --out ~/Downloads/Photos-3-001\(2\)/dr-lic-f.png/Users/lukas/Downloads/Photos-3-001(2)/dr-lic-f.HEIC/Users/lukas/Downloads/Photos-3-001(2)/dr-lic-f.pnglukas@Lukas-Kovaliks-MacBook-Pro-Jiminny~/jiminny/app (JY-20676-delete-report-related-objects)§ sips -sformat png ~/Downloads/Photos-3-001\(2\)/dr-lic-b.HEIC --out ~/Downloads/Photos-3-001\(2\)/dr-lic-b.png/Users/lukas/Downloads/Photos-3-001(2)/dr-lic-b.HEIC/Users/lukas/Downloads/Photos-3-001(2)/dr-lic-b.pnglukas@Lukas-Kovaliks-MacBook-Pro-Jiminny ~/jiminny/app (JY-20676-delete-report-related-objects)sipsformatpng ~/Downloads/Photos-3-001\(2\)/gtp.HEIC --out ~/Downloads/Photos-3-001\(2)/gtp.pngWarning: /Users/lukas/Downloads/Photos-3-001(2)/gtp.HEIC not a validfileskippinglukas@Lukas-Kovaliks-MacBook-Pro-Jiminny ~/jiminny/app (JY-20676-delete-report-related-objects)sips -sformatpng ~/Downloads/Photos-3-001\(2\)/gtp.heif --out ~/Downloads/Photos-3-001\(2\)/gtp.png/Users/lukas/Downloads/Photos-3-001(2)/gtp.heif/Users/lukas/Downloads/Photos-3-001(2)/gtp.pnglukas@Lukas-Kovaliks-MacBook-Pro-Jiminny~/jiminny/app (JY-20676-delete-report-related-objects) $ sipsformatpng ~/Downloads/Photos-3-001\(2\)/talon-m-f.HEIC --out ~/Downloads/Photos-3-001\(2\)/talon-m-f.png/Users/lukas/Downloads/Photos-3-001(2)/talon-m-f.HEIC/Users/lukas/Downloads/Photos-3-001(2)/talon-m-f.pngSlacklukas@Lukas-Kovaliks-MacBook-Pro-Jiminny ~/jiminny/app (JY-20676-delete-report-related-objects) $ sips-s format png ~/Downloads/Photos-3-001\(2\)/talon-m-b.HEIC --out ~/Downloads/Photos-3-001\(2\)/talon-m-b.png/Users/lukas/Downloads/Photos-3-001(2)/t‹$1PS/Users/lukas/Downloads/Photos-3-001(2),liollukas@Lukas-Kovaliks-MacBook-Pro-Jiminny...
|
NULL
|
500965305775279371
|
NULL
|
click
|
ocr
|
NULL
|
SlackFileEditViewGoHistoryWindowHelp> 0 lllDOCK SlackFileEditViewGoHistoryWindowHelp> 0 lllDOCKERO ₴1DEV (-zsh)APP (-zsh)|₴2screenpipe"20О 83A100% C4 8• Wed 20 May 18:29:32181APP (-zsh)Fixed 0 of 5690 files in 78.144 seconds, 60.00MBmemoryusedWhat's next:Try Docker Debug for seamless, persistentdebugging tools in any containeror image → docker debug docker_lamp_1Learn more at [URL_WITH_CREDENTIALS] (JY-20676-delete-report-related-objects) $ sips -s format png ~/Downloads/Photos-3-001\(2\)/talon-f.HEIC --out ~/Downloads/Photos-3-001\(2\)/talon-f.png/Users/lukas/Downloads/Photos-3-001(2)/talon-f.HEIC/Users/lukas/Downloads/Photos-3-001(2)/talon-f.pnglukas@Lukas-Kovaliks-MacBook-Pro-Jiminny ~/jiminny/app(JY-20676-delete-report-related-objects) $ sips -s format png ~/Downloads/Photos-3-001\(2\)/talon-b.HEIC --out ~/Downloads/Photos-3-001\(2\)/talon-b.png/Users/lukas/Downloads/Photos-3-001(2)/talon-b.HEIC/Users/lukas/Downloads/Photos-3-001(2)/talon-b.pnglukas@Lukas-Kovaliks-MacBook-Pro-Jiminny ~/jiminny/app(JY-20676-delete-report-related-objects) $ sips -s format png ~/Downloads/Photos-3-001\(2\)/vin.HEIC --out ~/Downloads/Photos-3-001\(2\)/vin.png/Users/lukas/Downloads/Photos-3-001(2)/vin.HEIC/Users/lukas/Downloads/Photos-3-001(2)/vin.pnglukas@Lukas-Kovaliks-MacBook-Pro-Jiminny ~/jiminny/app (JY-20676-delete-report-related-objects) $ sips -s format png ~/Downloads/Photos-3-001\(2\)/dr-lic-f.HEIC --out ~/Downloads/Photos-3-001\(2\)/dr-lic-f.png/Users/lukas/Downloads/Photos-3-001(2)/dr-lic-f.HEIC/Users/lukas/Downloads/Photos-3-001(2)/dr-lic-f.pnglukas@Lukas-Kovaliks-MacBook-Pro-Jiminny~/jiminny/app (JY-20676-delete-report-related-objects)§ sips -sformat png ~/Downloads/Photos-3-001\(2\)/dr-lic-b.HEIC --out ~/Downloads/Photos-3-001\(2\)/dr-lic-b.png/Users/lukas/Downloads/Photos-3-001(2)/dr-lic-b.HEIC/Users/lukas/Downloads/Photos-3-001(2)/dr-lic-b.pnglukas@Lukas-Kovaliks-MacBook-Pro-Jiminny ~/jiminny/app (JY-20676-delete-report-related-objects)sipsformatpng ~/Downloads/Photos-3-001\(2\)/gtp.HEIC --out ~/Downloads/Photos-3-001\(2)/gtp.pngWarning: /Users/lukas/Downloads/Photos-3-001(2)/gtp.HEIC not a validfileskippinglukas@Lukas-Kovaliks-MacBook-Pro-Jiminny ~/jiminny/app (JY-20676-delete-report-related-objects)sips -sformatpng ~/Downloads/Photos-3-001\(2\)/gtp.heif --out ~/Downloads/Photos-3-001\(2\)/gtp.png/Users/lukas/Downloads/Photos-3-001(2)/gtp.heif/Users/lukas/Downloads/Photos-3-001(2)/gtp.pnglukas@Lukas-Kovaliks-MacBook-Pro-Jiminny~/jiminny/app (JY-20676-delete-report-related-objects) $ sipsformatpng ~/Downloads/Photos-3-001\(2\)/talon-m-f.HEIC --out ~/Downloads/Photos-3-001\(2\)/talon-m-f.png/Users/lukas/Downloads/Photos-3-001(2)/talon-m-f.HEIC/Users/lukas/Downloads/Photos-3-001(2)/talon-m-f.pngSlacklukas@Lukas-Kovaliks-MacBook-Pro-Jiminny ~/jiminny/app (JY-20676-delete-report-related-objects) $ sips-s format png ~/Downloads/Photos-3-001\(2\)/talon-m-b.HEIC --out ~/Downloads/Photos-3-001\(2\)/talon-m-b.png/Users/lukas/Downloads/Photos-3-001(2)/t‹$1PS/Users/lukas/Downloads/Photos-3-001(2),liollukas@Lukas-Kovaliks-MacBook-Pro-Jiminny...
|
65210
|
NULL
|
NULL
|
NULL
|
|
65222
|
2286
|
20
|
2026-05-20T15:29:32.978570+00:00
|
/Users/lukas/.screenpipe/data/data/2026-05-20/1779 /Users/lukas/.screenpipe/data/data/2026-05-20/1779290972978_m2.jpg...
|
Firefox
|
Inbox - kovaliklukas@gmail.com - Gmail — Personal
|
1
|
mail.google.com/mail/u/0/#inbox
|
monitor_2
|
NULL
|
NULL
|
NULL
|
NULL
|
PhostormVIewINavigarecodeLaravelrTavsco.s#12105 on PhostormVIewINavigarecodeLaravelrTavsco.s#12105 on JY-20915-add-domain-Proiect© WebhookController.php• D ActivitiesNourications//smskecelveo.pngsms-received.blade.phpParticipantDeclinedRecordingTest.phy©smskecelvealest.ong>@ ActivityProvidersE.env.production› L AlAutomationclass SmsRecelvedlest extends TestcaseW Plavlistsprivate const string NOTIFIABLE_EMAIL = '[EMAIL]':© ActivityLiveCoachingNoteTest.phpc) ActivitvNotLoqgedTest.phopravace const scring buublelcx/nusl ="cxc.J1m1nny.com@ ActivityScheduledTest.php@ ActivityScoredTest.php9 NudgeExpirationWarningNotificationTest 29 €@ OpportunitvUpdateNotificationTest.php@ UserinvitedToTeamWithEmailOnlvTest.olprocecced tunccion secupo. voldparent::setUpo:Ooservers1 Policiesconf1q::set('11m1nnv.aoogle_text.host'. self::G00GLE_TEXTHOST*Providersconf1q::set('11m1nnv.deplovregzon', 'us')›|N RevositoriesM RulesM Services37 6tprotected function tearDown: voicLocal ChangesShelfConsoleLoaChandes 2 filesTJ9 <→₴.Side-by-side viewer -Do not ignoreE.env.local appnb8c8eb56 confia/loaaina.oho© JiminnyDebugCommand.php app/Console/Commandspnp logging.onp contieUnversioned Files 9 filesE.env.nikilocal app'Level" => env('LOG_LEVEL', 'info').nath' => storade nath'loas/laravel.100')E.env.other app© CanAccessAiReportsTest.php tests/Unit/Policies© CreateMockAskJiminnyReportResultCommand.php app/Console/Commands/Re( favicon.ico publicE ids.txt appTeraw so querv sal aopSimulateWebhooksCommand.phv app/Console/Commands/Crm/HubspoM. WEBHOOK FILTERING IMPLEMENTATION md aoo(©) TeytRelavService.png100% L2Wed 20 May 18:29:32+0 ..= custom.log= laravel.log4 SF jiminny@localhost]4 HS_local [jiminny@localhost]« console [PROD] X# console [euyA console [STAGING]10 ^v Q.664665— 666=667[CREDIT_CARD]- 676677678Tx: AutovPlaygroundvSo jiminny v:LECT * FRUM users WHERE 1d IN (24625.29445.29615971041 A1 A40 V65 ^ELECT * FROM automated_report_results WHERE uuid_to_bin('822fa41b-afd3-43a9-a248-86b0e36f3131') = uuid;elect * from text_relays where created_at > 2026-01-01';nd id IN (32415, 32416);and id= 324121elect * from users where team_id = 2 and email like '%scott%' and id = 29510;ELECT * FROM activities WHERE uuid_to_bin('67cebfc2-ed56-44a2-8c68-7a0286ed8618') = uuid; # 79763436ELECT email_provider_id, COUNT(*) as count, GROUP_CONCAT(id) as ids, GROUP_CONCAT(status) as statuses2OM text_relaysTERE email_provider_id IN ('19e2027868a64b42',ROUP BY email_provider_id;ELECT id, status, telephony_provider_id, created_at119e2033ed8ea6h10')CascadeImprove SmsReceived<truncatea 104 unes>HubSpot\CLient Webhooks \Model|SubscriptionResponse: :offsetUnset(Soffset) should eitherDeprecated: Return type of HubSpot Client Webhooks Model SubscriptionResponse::1sonSerialize• docker exec docker lamp 1 php artisan test --t1lter=TextRelayServiceresttruncated 168 1inessTestslu tserviceHpurent2m Doaste Veure tast coce d use aet Fibures Precest.te deprecated eld elet do lenger be supported inpAsk anything (&+L)< Code SWF.1.61 differencecurrent version'driver' => 'errorlog'.'Level' => env('LOG_LEVEL', 'info'),'path" => storage_path('Logs/Laravel.Log').'custom channel' => 1'driver' => 'single','path' => storage_path('logs/custom.log')."levelt =s envriind lEVElt Tinfar)8 Pushed 1 commit tooriain/JY-20915-add-domain-specific-email-textView null reauesA 2 files committedJY-20915 TIX testsEdit Commit Messade.Po. 4 space...
|
NULL
|
916533503881126676
|
NULL
|
click
|
ocr
|
NULL
|
PhostormVIewINavigarecodeLaravelrTavsco.s#12105 on PhostormVIewINavigarecodeLaravelrTavsco.s#12105 on JY-20915-add-domain-Proiect© WebhookController.php• D ActivitiesNourications//smskecelveo.pngsms-received.blade.phpParticipantDeclinedRecordingTest.phy©smskecelvealest.ong>@ ActivityProvidersE.env.production› L AlAutomationclass SmsRecelvedlest extends TestcaseW Plavlistsprivate const string NOTIFIABLE_EMAIL = '[EMAIL]':© ActivityLiveCoachingNoteTest.phpc) ActivitvNotLoqgedTest.phopravace const scring buublelcx/nusl ="cxc.J1m1nny.com@ ActivityScheduledTest.php@ ActivityScoredTest.php9 NudgeExpirationWarningNotificationTest 29 €@ OpportunitvUpdateNotificationTest.php@ UserinvitedToTeamWithEmailOnlvTest.olprocecced tunccion secupo. voldparent::setUpo:Ooservers1 Policiesconf1q::set('11m1nnv.aoogle_text.host'. self::G00GLE_TEXTHOST*Providersconf1q::set('11m1nnv.deplovregzon', 'us')›|N RevositoriesM RulesM Services37 6tprotected function tearDown: voicLocal ChangesShelfConsoleLoaChandes 2 filesTJ9 <→₴.Side-by-side viewer -Do not ignoreE.env.local appnb8c8eb56 confia/loaaina.oho© JiminnyDebugCommand.php app/Console/Commandspnp logging.onp contieUnversioned Files 9 filesE.env.nikilocal app'Level" => env('LOG_LEVEL', 'info').nath' => storade nath'loas/laravel.100')E.env.other app© CanAccessAiReportsTest.php tests/Unit/Policies© CreateMockAskJiminnyReportResultCommand.php app/Console/Commands/Re( favicon.ico publicE ids.txt appTeraw so querv sal aopSimulateWebhooksCommand.phv app/Console/Commands/Crm/HubspoM. WEBHOOK FILTERING IMPLEMENTATION md aoo(©) TeytRelavService.png100% L2Wed 20 May 18:29:32+0 ..= custom.log= laravel.log4 SF jiminny@localhost]4 HS_local [jiminny@localhost]« console [PROD] X# console [euyA console [STAGING]10 ^v Q.664665— 666=667[CREDIT_CARD]- 676677678Tx: AutovPlaygroundvSo jiminny v:LECT * FRUM users WHERE 1d IN (24625.29445.29615971041 A1 A40 V65 ^ELECT * FROM automated_report_results WHERE uuid_to_bin('822fa41b-afd3-43a9-a248-86b0e36f3131') = uuid;elect * from text_relays where created_at > 2026-01-01';nd id IN (32415, 32416);and id= 324121elect * from users where team_id = 2 and email like '%scott%' and id = 29510;ELECT * FROM activities WHERE uuid_to_bin('67cebfc2-ed56-44a2-8c68-7a0286ed8618') = uuid; # 79763436ELECT email_provider_id, COUNT(*) as count, GROUP_CONCAT(id) as ids, GROUP_CONCAT(status) as statuses2OM text_relaysTERE email_provider_id IN ('19e2027868a64b42',ROUP BY email_provider_id;ELECT id, status, telephony_provider_id, created_at119e2033ed8ea6h10')CascadeImprove SmsReceived<truncatea 104 unes>HubSpot\CLient Webhooks \Model|SubscriptionResponse: :offsetUnset(Soffset) should eitherDeprecated: Return type of HubSpot Client Webhooks Model SubscriptionResponse::1sonSerialize• docker exec docker lamp 1 php artisan test --t1lter=TextRelayServiceresttruncated 168 1inessTestslu tserviceHpurent2m Doaste Veure tast coce d use aet Fibures Precest.te deprecated eld elet do lenger be supported inpAsk anything (&+L)< Code SWF.1.61 differencecurrent version'driver' => 'errorlog'.'Level' => env('LOG_LEVEL', 'info'),'path" => storage_path('Logs/Laravel.Log').'custom channel' => 1'driver' => 'single','path' => storage_path('logs/custom.log')."levelt =s envriind lEVElt Tinfar)8 Pushed 1 commit tooriain/JY-20915-add-domain-specific-email-textView null reauesA 2 files committedJY-20915 TIX testsEdit Commit Messade.Po. 4 space...
|
NULL
|
NULL
|
NULL
|
NULL
|
|
65226
|
NULL
|
0
|
2026-05-20T15:29:44.367438+00:00
|
/Users/lukas/.screenpipe/data/data/2026-05-20/1779 /Users/lukas/.screenpipe/data/data/2026-05-20/1779290984367_m1.jpg...
|
Firefox
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira — Work...
|
1
|
jiminny.atlassian.net/jira/software/c/projects/JY/ jiminny.atlassian.net/jira/software/c/projects/JY/boards/37...
|
monitor_1
|
NULL
|
NULL
|
NULL
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Close tab
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo
Share & export
Copy
Show more options
Ask Gemini
Ask Gemini
Upload & tools
Microphone
Send message
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Summarize page
Summarize page
Skip to:
Top Bar
Top Bar
Sidebar
Sidebar
Main Content
Main Content
Space navigation
Space navigation
Collapse sidebar [
Collapse sidebar [
Switch sites or apps
Switch sites or apps
Go to your Jira homepage
Search, press enter to navigate to advanced search with your text query
Create
Create
Rovo Ask Rovo
Ask Rovo
Over 9 Notifications
Over 9 Notifications
Help
Help
Settings
Settings...
|
[{"role":"AXRadioButton","text [{"role":"AXRadioButton","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Close tab","depth":5,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXRadioButton","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Pipelines - jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Pipelines - jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Tab","depth":4,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Customize sidebar","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Close Google Gemini (⌃X)","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Tabs from other devices","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open history (⇧⌘H)","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open bookmarks (⌘B)","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"AI Chat settings","depth":7,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Close","depth":7,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Main menu","depth":13,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"New chat","depth":12,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Gemini","depth":15,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Chat","depth":13,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Open menu for conversation actions.","depth":13,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Conversation with Gemini","depth":15,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Conversation with Gemini","depth":16,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"You said I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.","depth":23,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Expand","depth":21,"on_screen":false,"role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a concise summary of the changes made in the code selection:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Refactoring and Dependencies","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Refactoring and Dependencies","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Replaced Interface with Implementation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Changed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusInterface","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusManager","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"in the imports and constructor dependency injection.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Redis Support:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Imported","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Illuminate\\Support\\Facades\\Redis","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to handle a newly introduced memory-logging lock mechanism.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Traits:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"WorkerIdTrait","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", along with the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"worker","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"field tracking in the final logs.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Loop and Worker Lifecycle Adjustments","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Loop and Worker Lifecycle Adjustments","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Worker Auto-Restart:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Deleted the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property, the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"constant, and the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"shouldIterateAgain()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method. The script no longer tracks execution iterations or triggers a self-managed worker restart.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Simplified Chunk Method Return:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Modified","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to always return","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Priority Handling for Throttled Entities","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Priority Handling for Throttled Entities","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Stale Record Elevation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Updated the logic in","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"rescheduleThrottledEntities()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to check if a throttled entity group is classified as \"stale\". If it is, the entities are rescheduled using","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"QueuePriorityEnum::NORMAL","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"; otherwise, they keep their respective queue priority.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Metrics, Logging, and Environment Handling","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Metrics, Logging, and Environment Handling","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Environment Tracking:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Introduced a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$environment","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property (defaulting to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"'local'","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") and a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"setEnvironment()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method to dynamically assign it.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Memory Monitoring via Datadog Gauge:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method that submits peak memory allocation to Datadog as a gauge metric (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"jiminny.reindex-memory-usage","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis Lock for Metrics Throttling:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Implemented a Redis-backed lock (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"%s-process-manager-lock","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") with a 60-second expiration inside","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to prevent over-reporting metrics.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Log Payload Update:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added total processed items (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$updatedDocuments + $deletedDocuments","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") to the final information log.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":23,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"You said are there any issues or potential bottlenecks","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"are there any issues or potential bottlenecks","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Edit","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a breakdown of the potential issues to look out for:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"1. Memory Leak Risk (OOM Crashes)","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"1. Memory Leak Risk (OOM Crashes)","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Restart Logic:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The most significant change is the removal of the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Dependency Leaks:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If any underlying service (like the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ElasticSearchClient","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", ORM, or Datadog helper) retains references to processed entities,","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Hardcoded Limits:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The code still includes","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ini_set('memory_limit', '512M');","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":". Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"2. Race Condition in Redis Lock","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"2. Race Condition in Redis Lock","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Non-Atomic Operations:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method uses three distinct steps to handle the lock:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"get()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":",","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", and","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"expire()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If multiple workers process the same","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$this->updateTarget","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"concurrently, they can both pass the","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"if (! is_null(Redis::get(...)))","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"check at the exact same millisecond. Both will then execute","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and push duplicate metrics to Datadog.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Fix:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"This should be replaced with a single atomic operation using Redis's","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"NX","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Not eXists) and","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"EX","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Expire) flags, or Laravel’s built-in atomic cache locks:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis::set($gaugeLockName, true, 'EX', 60, 'NX')","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"3. Infinite Loops and Caller Contract","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"3. Infinite Loops and Caller Contract","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Return Value Change:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"previously returned whether it should iterate again based on staleness or chunk size. It now always returns","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(unless the selection is totally empty).","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the code calling this method relies on a","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"false","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"return to pause, sleep, or gracefully terminate, changing this to an unconditional","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"could result in an aggressive \"busy loop\" that spikes CPU usage.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"4. Performance Overhead","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"4. Performance Overhead","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Expensive Garbage Collection:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Calling","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles();","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"on","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"every single chunk","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Synchronous Metric Dispatch:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::increment","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"or","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::gauge","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"5. Lack of Explicit Error Handling","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5. Lack of Explicit Error Handling","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"No","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"try/catch","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Blocks:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The script executes complex network operations (","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"loadDocuments","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":22,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":22,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Redo","depth":22,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":21,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":22,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":21,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXTextArea","text":"Ask Gemini","depth":21,"on_screen":true,"value":"Ask Gemini","help_text":"","role_description":"text entry area","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Ask Gemini","depth":22,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Upload & tools","depth":20,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Microphone","depth":20,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Send message","depth":20,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":false,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.","depth":17,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Your privacy & Gemini Opens in a new window","depth":17,"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Your privacy & Gemini","depth":18,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Opens in a new window","depth":19,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Summarize page","depth":7,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Summarize page","depth":9,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Skip to:","depth":9,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Top Bar","depth":10,"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Top Bar","depth":11,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Sidebar","depth":10,"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Sidebar","depth":11,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Main Content","depth":10,"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Main Content","depth":11,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Space navigation","depth":10,"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Space navigation","depth":11,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Collapse sidebar [","depth":9,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Collapse sidebar [","depth":11,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXMenuButton","text":"Switch sites or apps","depth":10,"on_screen":true,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXStaticText","text":"Switch sites or apps","depth":12,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Go to your Jira homepage","depth":9,"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXComboBox","text":"Search, press enter to navigate to advanced search with your text query","depth":11,"on_screen":true,"help_text":"","placeholder":"Search","role_description":"combo box","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Create","depth":10,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Create","depth":12,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Rovo Ask Rovo","depth":12,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Ask Rovo","depth":14,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Over 9 Notifications","depth":12,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Over 9 Notifications","depth":14,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Help","depth":12,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Help","depth":14,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXMenuButton","text":"Settings","depth":12,"on_screen":true,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXStaticText","text":"Settings","depth":14,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"}]...
|
8807987019575634436
|
-1052937846671444068
|
visual_change
|
accessibility
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Close tab
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo
Share & export
Copy
Show more options
Ask Gemini
Ask Gemini
Upload & tools
Microphone
Send message
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Summarize page
Summarize page
Skip to:
Top Bar
Top Bar
Sidebar
Sidebar
Main Content
Main Content
Space navigation
Space navigation
Collapse sidebar [
Collapse sidebar [
Switch sites or apps
Switch sites or apps
Go to your Jira homepage
Search, press enter to navigate to advanced search with your text query
Create
Create
Rovo Ask Rovo
Ask Rovo
Over 9 Notifications
Over 9 Notifications
Help
Help
Settings
Settings...
|
NULL
|
NULL
|
NULL
|
NULL
|
|
65227
|
2286
|
21
|
2026-05-20T15:29:46.710392+00:00
|
/Users/lukas/.screenpipe/data/data/2026-05-20/1779 /Users/lukas/.screenpipe/data/data/2026-05-20/1779290986710_m2.jpg...
|
Firefox
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira — Work...
|
1
|
jiminny.atlassian.net/jira/software/c/projects/JY/ jiminny.atlassian.net/jira/software/c/projects/JY/boards/37...
|
monitor_2
|
NULL
|
NULL
|
NULL
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Close tab
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo
Share & export
Copy
Show more options
Ask Gemini
Ask Gemini
Upload & tools
Microphone
Send message
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Summarize page
Summarize page
Skip to:
Top Bar
Top Bar
Sidebar
Sidebar
Main Content
Main Content
Space navigation
Space navigation
Collapse sidebar [
Collapse sidebar [
Switch sites or apps
Switch sites or apps
Go to your Jira homepage
Search, press enter to navigate to advanced search with your text query
Create
Create
Rovo Ask Rovo
Ask Rovo
Over 9 Notifications
Over 9 Notifications
Help
Help
Settings
Settings
[EMAIL]
[EMAIL]
For you
For you
Recent
Recent
Starred
Starred
Apps
Apps
More actions for Apps
More actions for Apps
Spaces
Spaces
Create space
Create space
More actions for spaces
More actions for spaces
Recent
Jiminny (New)
Jiminny (New)
Jiminny (New)
Create board
Create board
More actions for Jiminny (New)
More actions for Jiminny (New)
Platform Team
Platform Team
Board actions
Board actions
Capture Team
Capture Team
Board actions
Board actions
Enterprise Stability Issues 🤕
Enterprise Stability Issues 🤕
Board actions
Board actions
Processing Team
Processing Team
Board actions
Board actions
SE Kanban
SE Kanban
Board actions
Board actions
Service-Desk
Service-Desk
More actions for Service-Desk...
|
[{"role":"AXRadioButton","text [{"role":"AXRadioButton","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":4,"bounds":{"left":0.0,"top":0.0518755,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.06304868,"width":0.10106383,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Close tab","depth":5,"bounds":{"left":0.06732048,"top":0.05905826,"width":0.007978723,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXRadioButton","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":4,"bounds":{"left":0.0,"top":0.08459697,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.09577015,"width":0.10721409,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.11731844,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.12849163,"width":0.15791224,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.15003991,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.16121309,"width":0.1796875,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.18276137,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.19393456,"width":0.12699468,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":4,"bounds":{"left":0.0,"top":0.21548285,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":5,"bounds":{"left":0.013297873,"top":0.22665602,"width":0.10139628,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":4,"bounds":{"left":0.0,"top":0.2482043,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.25937748,"width":0.18068483,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":4,"bounds":{"left":0.0,"top":0.28092578,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":5,"bounds":{"left":0.013297873,"top":0.29209897,"width":0.4644282,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Pipelines - jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.31364724,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Pipelines - jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.32482043,"width":0.039228722,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.3463687,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.3575419,"width":0.15824468,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Tab","depth":4,"bounds":{"left":0.0028257978,"top":0.38068634,"width":0.07413564,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Customize sidebar","depth":6,"bounds":{"left":0.0028257978,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Close Google Gemini (⌃X)","depth":6,"bounds":{"left":0.013796543,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Tabs from other devices","depth":6,"bounds":{"left":0.024933511,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open history (⇧⌘H)","depth":6,"bounds":{"left":0.036070477,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open bookmarks (⌘B)","depth":6,"bounds":{"left":0.04720745,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"AI Chat settings","depth":7,"bounds":{"left":0.35854387,"top":0.055067837,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Close","depth":7,"bounds":{"left":0.37051198,"top":0.055067837,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Main menu","depth":13,"bounds":{"left":0.08228058,"top":0.09736632,"width":0.015957447,"height":0.03830806},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"New chat","depth":12,"bounds":{"left":0.09823803,"top":0.10215483,"width":0.028091755,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Gemini","depth":15,"bounds":{"left":0.1008976,"top":0.10654429,"width":0.02144282,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Chat","depth":13,"bounds":{"left":0.35854387,"top":0.10215483,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Open menu for conversation actions.","depth":13,"bounds":{"left":0.37051198,"top":0.10215483,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Conversation with Gemini","depth":15,"bounds":{"left":0.079288565,"top":0.14445332,"width":0.0003324468,"height":0.0007980846},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Conversation with Gemini","depth":16,"bounds":{"left":0.079288565,"top":0.14684756,"width":0.1200133,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"You said I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.","depth":23,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Expand","depth":21,"on_screen":false,"role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a concise summary of the changes made in the code selection:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Refactoring and Dependencies","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Refactoring and Dependencies","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Replaced Interface with Implementation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Changed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusInterface","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusManager","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"in the imports and constructor dependency injection.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Redis Support:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Imported","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Illuminate\\Support\\Facades\\Redis","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to handle a newly introduced memory-logging lock mechanism.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Traits:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"WorkerIdTrait","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", along with the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"worker","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"field tracking in the final logs.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Loop and Worker Lifecycle Adjustments","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Loop and Worker Lifecycle Adjustments","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Worker Auto-Restart:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Deleted the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property, the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"constant, and the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"shouldIterateAgain()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method. The script no longer tracks execution iterations or triggers a self-managed worker restart.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Simplified Chunk Method Return:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Modified","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to always return","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Priority Handling for Throttled Entities","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Priority Handling for Throttled Entities","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Stale Record Elevation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Updated the logic in","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"rescheduleThrottledEntities()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to check if a throttled entity group is classified as \"stale\". If it is, the entities are rescheduled using","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"QueuePriorityEnum::NORMAL","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"; otherwise, they keep their respective queue priority.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Metrics, Logging, and Environment Handling","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Metrics, Logging, and Environment Handling","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Environment Tracking:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Introduced a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$environment","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property (defaulting to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"'local'","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") and a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"setEnvironment()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method to dynamically assign it.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Memory Monitoring via Datadog Gauge:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method that submits peak memory allocation to Datadog as a gauge metric (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"jiminny.reindex-memory-usage","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis Lock for Metrics Throttling:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Implemented a Redis-backed lock (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"%s-process-manager-lock","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") with a 60-second expiration inside","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to prevent over-reporting metrics.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Log Payload Update:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added total processed items (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$updatedDocuments + $deletedDocuments","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") to the final information log.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":23,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"You said are there any issues or potential bottlenecks","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"are there any issues or potential bottlenecks","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Edit","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a breakdown of the potential issues to look out for:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"1. Memory Leak Risk (OOM Crashes)","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"1. Memory Leak Risk (OOM Crashes)","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Restart Logic:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The most significant change is the removal of the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Dependency Leaks:","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.050033245,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If any underlying service (like the","depth":26,"bounds":{"left":0.17453457,"top":0.0,"width":0.08277926,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ElasticSearchClient","depth":27,"bounds":{"left":0.25930852,"top":0.0,"width":0.056848403,"height":0.015163607},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", ORM, or Datadog helper) retains references to processed entities,","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.21775267,"height":0.03631285},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"bounds":{"left":0.26811835,"top":0.0,"width":0.056848403,"height":0.015163607},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.23138298,"height":0.05546688},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Hardcoded Limits:","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.047041222,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The code still includes","depth":26,"bounds":{"left":0.17154256,"top":0.0,"width":0.05651596,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ini_set('memory_limit', '512M');","depth":27,"bounds":{"left":0.23005319,"top":0.0,"width":0.095744684,"height":0.015163607},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":". Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.23055187,"height":0.05546688},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"2. Race Condition in Redis Lock","depth":23,"bounds":{"left":0.111369684,"top":0.07940942,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"2. Race Condition in Redis Lock","depth":24,"bounds":{"left":0.111369684,"top":0.079010375,"width":0.09125665,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Non-Atomic Operations:","depth":26,"bounds":{"left":0.12450133,"top":0.10614525,"width":0.062333778,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The","depth":26,"bounds":{"left":0.18683511,"top":0.10614525,"width":0.011635638,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage","depth":27,"bounds":{"left":0.20046543,"top":0.10774142,"width":0.041888297,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method uses three distinct steps to handle the lock:","depth":26,"bounds":{"left":0.12450133,"top":0.10614525,"width":0.22722739,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"get()","depth":27,"bounds":{"left":0.14860372,"top":0.12689546,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":",","depth":26,"bounds":{"left":0.16555852,"top":0.12529927,"width":0.002493351,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"bounds":{"left":0.17004654,"top":0.12689546,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", and","depth":26,"bounds":{"left":0.18700133,"top":0.12529927,"width":0.013131649,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"expire()","depth":27,"bounds":{"left":0.20212767,"top":0.12689546,"width":0.023936171,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"bounds":{"left":0.22805852,"top":0.12529927,"width":0.0013297872,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"bounds":{"left":0.12450133,"top":0.15403032,"width":0.040059842,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If multiple workers process the same","depth":26,"bounds":{"left":0.16456117,"top":0.15403032,"width":0.09275266,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$this->updateTarget","depth":27,"bounds":{"left":0.25930852,"top":0.15562649,"width":0.056848403,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"concurrently, they can both pass the","depth":26,"bounds":{"left":0.12450133,"top":0.15403032,"width":0.22855718,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"if (! is_null(Redis::get(...)))","depth":27,"bounds":{"left":0.18351063,"top":0.17478053,"width":0.09275266,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"check at the exact same millisecond. Both will then execute","depth":26,"bounds":{"left":0.12450133,"top":0.17318435,"width":0.21575798,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"bounds":{"left":0.2122673,"top":0.19393456,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and push duplicate metrics to Datadog.","depth":26,"bounds":{"left":0.22922207,"top":0.19233839,"width":0.09823803,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Fix:","depth":26,"bounds":{"left":0.12450133,"top":0.22106944,"width":0.019281914,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"This should be replaced with a single atomic operation using Redis's","depth":26,"bounds":{"left":0.14378324,"top":0.22106944,"width":0.16821809,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"NX","depth":27,"bounds":{"left":0.31399602,"top":0.22266561,"width":0.005984043,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Not eXists) and","depth":26,"bounds":{"left":0.12450133,"top":0.22106944,"width":0.22855718,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"EX","depth":27,"bounds":{"left":0.13696809,"top":0.24181964,"width":0.005984043,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Expire) flags, or Laravel’s built-in atomic cache locks:","depth":26,"bounds":{"left":0.14494681,"top":0.24022347,"width":0.13264628,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis::set($gaugeLockName, true, 'EX', 60, 'NX')","depth":27,"bounds":{"left":0.12649602,"top":0.26097366,"width":0.14361702,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"bounds":{"left":0.27210772,"top":0.25937748,"width":0.0013297872,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"3. Infinite Loops and Caller Contract","depth":23,"bounds":{"left":0.111369684,"top":0.30925778,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"3. Infinite Loops and Caller Contract","depth":24,"bounds":{"left":0.111369684,"top":0.30885875,"width":0.10571808,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Return Value Change:","depth":26,"bounds":{"left":0.12450133,"top":0.33599362,"width":0.055352394,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"bounds":{"left":0.18301196,"top":0.33758977,"width":0.06881649,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"previously returned whether it should iterate again based on staleness or chunk size. It now always returns","depth":26,"bounds":{"left":0.12450133,"top":0.33599362,"width":0.2237367,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"bounds":{"left":0.29438165,"top":0.3567438,"width":0.011968086,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(unless the selection is totally empty).","depth":26,"bounds":{"left":0.12450133,"top":0.35514766,"width":0.2130984,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"bounds":{"left":0.12450133,"top":0.40303272,"width":0.040059842,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the code calling this method relies on a","depth":26,"bounds":{"left":0.16456117,"top":0.40303272,"width":0.10305851,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"false","depth":27,"bounds":{"left":0.26961437,"top":0.4046289,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"return to pause, sleep, or gracefully terminate, changing this to an unconditional","depth":26,"bounds":{"left":0.12450133,"top":0.40303272,"width":0.22589761,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"bounds":{"left":0.26163563,"top":0.4237829,"width":0.011968086,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"could result in an aggressive \"busy loop\" that spikes CPU usage.","depth":26,"bounds":{"left":0.12450133,"top":0.42218676,"width":0.22290559,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"4. Performance Overhead","depth":23,"bounds":{"left":0.111369684,"top":0.49122107,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"4. Performance Overhead","depth":24,"bounds":{"left":0.111369684,"top":0.49082202,"width":0.076130316,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Expensive Garbage Collection:","depth":26,"bounds":{"left":0.12450133,"top":0.5179569,"width":0.078457445,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Calling","depth":26,"bounds":{"left":0.20295878,"top":0.5179569,"width":0.019281914,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles();","depth":27,"bounds":{"left":0.22423537,"top":0.51955307,"width":0.059840426,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"on","depth":26,"bounds":{"left":0.28607047,"top":0.5179569,"width":0.00880984,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"every single chunk","depth":26,"bounds":{"left":0.29488033,"top":0.5179569,"width":0.045545213,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.","depth":26,"bounds":{"left":0.12450133,"top":0.5179569,"width":0.23420878,"height":0.05546688},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Synchronous Metric Dispatch:","depth":26,"bounds":{"left":0.12450133,"top":0.584996,"width":0.07712766,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the","depth":26,"bounds":{"left":0.20162898,"top":0.584996,"width":0.015292553,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::increment","depth":27,"bounds":{"left":0.21891622,"top":0.5865922,"width":0.053856384,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"or","depth":26,"bounds":{"left":0.27476728,"top":0.584996,"width":0.0078125,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::gauge","depth":27,"bounds":{"left":0.28457448,"top":0.5865922,"width":0.041888297,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.","depth":26,"bounds":{"left":0.12450133,"top":0.584996,"width":0.23420878,"height":0.05546688},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"5. Lack of Explicit Error Handling","depth":23,"bounds":{"left":0.111369684,"top":0.67318434,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5. Lack of Explicit Error Handling","depth":24,"bounds":{"left":0.111369684,"top":0.67278534,"width":0.094913565,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"No","depth":26,"bounds":{"left":0.12450133,"top":0.6999202,"width":0.008477394,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"try/catch","depth":27,"bounds":{"left":0.1349734,"top":0.7015164,"width":0.026928192,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Blocks:","depth":26,"bounds":{"left":0.16389628,"top":0.6999202,"width":0.019614361,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The script executes complex network operations (","depth":26,"bounds":{"left":0.18351063,"top":0.6999202,"width":0.12367021,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"loadDocuments","depth":27,"bounds":{"left":0.30917552,"top":0.7015164,"width":0.038896278,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.","depth":26,"bounds":{"left":0.12450133,"top":0.6999202,"width":0.22888963,"height":0.09377494},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":22,"bounds":{"left":0.1087101,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":22,"bounds":{"left":0.124667555,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Redo","depth":22,"bounds":{"left":0.140625,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":21,"bounds":{"left":0.15658244,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":22,"bounds":{"left":0.17253989,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":21,"bounds":{"left":0.18849733,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXTextArea","text":"Ask Gemini","depth":21,"bounds":{"left":0.14594415,"top":0.86632085,"width":0.17021276,"height":0.01915403},"on_screen":true,"value":"Ask Gemini","help_text":"","role_description":"text entry area","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Ask Gemini","depth":22,"bounds":{"left":0.14594415,"top":0.8667199,"width":0.030418882,"height":0.018355945},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Upload & tools","depth":20,"bounds":{"left":0.12732713,"top":0.8567438,"width":0.015957447,"height":0.03830806},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Microphone","depth":20,"bounds":{"left":0.32413563,"top":0.86153233,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Send message","depth":20,"bounds":{"left":0.33211437,"top":0.8631285,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":false,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.","depth":17,"bounds":{"left":0.102726065,"top":0.9173983,"width":0.21775267,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Your privacy & Gemini Opens in a new window","depth":17,"bounds":{"left":0.32047874,"top":0.9173983,"width":0.043882977,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Your privacy & Gemini","depth":18,"bounds":{"left":0.32047874,"top":0.9173983,"width":0.043882977,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Opens in a new window","depth":19,"bounds":{"left":0.079288565,"top":0.91660017,"width":0.04720745,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Summarize page","depth":7,"bounds":{"left":0.08494016,"top":0.95730245,"width":0.053523935,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Summarize page","depth":9,"bounds":{"left":0.09059176,"top":0.96249,"width":0.042220745,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Skip to:","depth":9,"bounds":{"left":0.39993352,"top":0.07861133,"width":0.016954787,"height":0.01396648},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Top Bar","depth":10,"bounds":{"left":0.39993352,"top":0.097765364,"width":0.016954787,"height":0.01396648},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Top Bar","depth":11,"bounds":{"left":0.39993352,"top":0.097765364,"width":0.016954787,"height":0.01396648},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Sidebar","depth":10,"bounds":{"left":0.39993352,"top":0.11691939,"width":0.016954787,"height":0.01396648},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Sidebar","depth":11,"bounds":{"left":0.39993352,"top":0.11691939,"width":0.016954787,"height":0.01396648},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Main Content","depth":10,"bounds":{"left":0.39993352,"top":0.13607343,"width":0.029421542,"height":0.01396648},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Main Content","depth":11,"bounds":{"left":0.39993352,"top":0.13607343,"width":0.029421542,"height":0.01396648},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Space navigation","depth":10,"bounds":{"left":0.39993352,"top":0.15522745,"width":0.037898935,"height":0.01396648},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Space navigation","depth":11,"bounds":{"left":0.39993352,"top":0.15522745,"width":0.037898935,"height":0.01396648},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Collapse sidebar [","depth":9,"bounds":{"left":0.3932846,"top":0.057861134,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Collapse sidebar [","depth":11,"bounds":{"left":0.3984375,"top":0.06344773,"width":0.039727394,"height":0.01396648},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXMenuButton","text":"Switch sites or apps","depth":10,"bounds":{"left":0.40525267,"top":0.057861134,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXStaticText","text":"Switch sites or apps","depth":12,"bounds":{"left":0.41040558,"top":0.06344773,"width":0.044215426,"height":0.01396648},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Go to your Jira homepage","depth":9,"bounds":{"left":0.41855052,"top":0.057861134,"width":0.029421542,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXComboBox","text":"Search, press enter to navigate to advanced search with your text query","depth":11,"bounds":{"left":0.5596742,"top":0.06264964,"width":0.24268617,"height":0.015961692},"on_screen":true,"help_text":"","placeholder":"Search","role_description":"combo box","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Create","depth":10,"bounds":{"left":0.81067157,"top":0.057861134,"width":0.030086435,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Create","depth":12,"bounds":{"left":0.82197475,"top":0.06384677,"width":0.014793883,"height":0.01396648},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Rovo Ask Rovo","depth":12,"bounds":{"left":0.91240025,"top":0.057861134,"width":0.035904255,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Ask Rovo","depth":14,"bounds":{"left":0.92370343,"top":0.06384677,"width":0.020611702,"height":0.01396648},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Over 9 Notifications","depth":12,"bounds":{"left":0.9496343,"top":0.057861134,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Over 9 Notifications","depth":14,"bounds":{"left":0.95478725,"top":0.06344773,"width":0.043882977,"height":0.01396648},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Help","depth":12,"bounds":{"left":0.9616024,"top":0.057861134,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Help","depth":14,"bounds":{"left":0.96675533,"top":0.06344773,"width":0.010139627,"height":0.01396648},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXMenuButton","text":"Settings","depth":12,"bounds":{"left":0.97357047,"top":0.057861134,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXStaticText","text":"Settings","depth":14,"bounds":{"left":0.9787234,"top":0.06344773,"width":0.017952127,"height":0.01396648},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXMenuButton","text":"lukas.kovalik@jiminny.com","depth":12,"bounds":{"left":0.98553854,"top":0.057861134,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXStaticText","text":"lukas.kovalik@jiminny.com","depth":14,"bounds":{"left":0.9906915,"top":0.06344773,"width":0.009308517,"height":0.01396648},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"For you","depth":12,"bounds":{"left":0.3932846,"top":0.09976058,"width":0.071476065,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"For you","depth":15,"bounds":{"left":0.4039229,"top":0.10574621,"width":0.01662234,"height":0.01396648},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Recent","depth":12,"bounds":{"left":0.3932846,"top":0.12529927,"width":0.071476065,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXStaticText","text":"Recent","depth":15,"bounds":{"left":0.4039229,"top":0.13128492,"width":0.015458777,"height":0.01396648},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Starred","depth":12,"bounds":{"left":0.3932846,"top":0.15083799,"width":0.071476065,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXStaticText","text":"Starred","depth":15,"bounds":{"left":0.4039229,"top":0.15682362,"width":0.016456118,"height":0.01396648},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Apps","depth":12,"bounds":{"left":0.3932846,"top":0.1763767,"width":0.071476065,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXStaticText","text":"Apps","depth":15,"bounds":{"left":0.4039229,"top":0.18236233,"width":0.011635638,"height":0.01396648},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXMenuButton","text":"More actions for Apps","depth":13,"bounds":{"left":0.46276596,"top":0.17956904,"width":0.0039893617,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXStaticText","text":"More actions for Apps","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Spaces","depth":12,"bounds":{"left":0.3932846,"top":0.2019154,"width":0.071476065,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":true},{"role":"AXStaticText","text":"Spaces","depth":15,"bounds":{"left":0.4039229,"top":0.20790103,"width":0.016456118,"height":0.01396648},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Create space","depth":13,"bounds":{"left":0.44614363,"top":0.20510775,"width":0.007978723,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Create space","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXMenuButton","text":"More actions for spaces","depth":13,"bounds":{"left":0.4554521,"top":0.20510775,"width":0.007978723,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXStaticText","text":"More actions for spaces","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Recent","depth":16,"bounds":{"left":0.39926863,"top":0.23423783,"width":0.013464096,"height":0.011971269},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Jiminny (New)","depth":17,"bounds":{"left":0.39727393,"top":0.2529928,"width":0.0674867,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Jiminny (New)","depth":20,"bounds":{"left":0.40791222,"top":0.25897846,"width":0.032081116,"height":0.01396648},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Jiminny (New)","depth":18,"bounds":{"left":0.39860374,"top":0.25618514,"width":0.007978723,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":true},{"role":"AXMenuButton","text":"Create board","depth":18,"bounds":{"left":0.44614363,"top":0.25618514,"width":0.007978723,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXStaticText","text":"Create board","depth":20,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXMenuButton","text":"More actions for Jiminny (New)","depth":18,"bounds":{"left":0.4554521,"top":0.25618514,"width":0.007978723,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXStaticText","text":"More actions for Jiminny (New)","depth":20,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Platform Team","depth":19,"bounds":{"left":0.4012633,"top":0.27853152,"width":0.06349734,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Platform Team","depth":22,"bounds":{"left":0.4119016,"top":0.28451717,"width":0.032247342,"height":0.01396648},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXMenuButton","text":"Board actions","depth":20,"bounds":{"left":0.46276596,"top":0.28172386,"width":0.0039893617,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXStaticText","text":"Board actions","depth":22,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Capture Team","depth":19,"bounds":{"left":0.4012633,"top":0.30407023,"width":0.06349734,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Capture Team","depth":22,"bounds":{"left":0.4119016,"top":0.31005585,"width":0.03125,"height":0.01396648},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXMenuButton","text":"Board actions","depth":20,"bounds":{"left":0.46276596,"top":0.30726257,"width":0.0039893617,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXStaticText","text":"Board actions","depth":22,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Enterprise Stability Issues 🤕","depth":19,"bounds":{"left":0.4012633,"top":0.32960895,"width":0.06349734,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Enterprise Stability Issues 🤕","depth":22,"bounds":{"left":0.4119016,"top":0.33559456,"width":0.050531916,"height":0.030726258},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXMenuButton","text":"Board actions","depth":20,"bounds":{"left":0.46276596,"top":0.33280128,"width":0.0039893617,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXStaticText","text":"Board actions","depth":22,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Processing Team","depth":19,"bounds":{"left":0.4012633,"top":0.35514766,"width":0.06349734,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Processing Team","depth":22,"bounds":{"left":0.4119016,"top":0.36113328,"width":0.038231384,"height":0.01396648},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXMenuButton","text":"Board actions","depth":20,"bounds":{"left":0.46276596,"top":0.35834,"width":0.0039893617,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXStaticText","text":"Board actions","depth":22,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"SE Kanban","depth":19,"bounds":{"left":0.4012633,"top":0.38068634,"width":0.06349734,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"SE Kanban","depth":22,"bounds":{"left":0.4119016,"top":0.386672,"width":0.024102394,"height":0.01396648},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXMenuButton","text":"Board actions","depth":20,"bounds":{"left":0.46276596,"top":0.38387868,"width":0.0039893617,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXStaticText","text":"Board actions","depth":22,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Service-Desk","depth":17,"bounds":{"left":0.39727393,"top":0.40622506,"width":0.0674867,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXStaticText","text":"Service-Desk","depth":20,"bounds":{"left":0.40791222,"top":0.4122107,"width":0.03025266,"height":0.01396648},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXMenuButton","text":"More actions for Service-Desk","depth":18,"bounds":{"left":0.46409574,"top":0.4094174,"width":0.0039893617,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false}]...
|
-8615738567144104846
|
-1012404900269295844
|
visual_change
|
accessibility
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Close tab
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo
Share & export
Copy
Show more options
Ask Gemini
Ask Gemini
Upload & tools
Microphone
Send message
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Summarize page
Summarize page
Skip to:
Top Bar
Top Bar
Sidebar
Sidebar
Main Content
Main Content
Space navigation
Space navigation
Collapse sidebar [
Collapse sidebar [
Switch sites or apps
Switch sites or apps
Go to your Jira homepage
Search, press enter to navigate to advanced search with your text query
Create
Create
Rovo Ask Rovo
Ask Rovo
Over 9 Notifications
Over 9 Notifications
Help
Help
Settings
Settings
[EMAIL]
[EMAIL]
For you
For you
Recent
Recent
Starred
Starred
Apps
Apps
More actions for Apps
More actions for Apps
Spaces
Spaces
Create space
Create space
More actions for spaces
More actions for spaces
Recent
Jiminny (New)
Jiminny (New)
Jiminny (New)
Create board
Create board
More actions for Jiminny (New)
More actions for Jiminny (New)
Platform Team
Platform Team
Board actions
Board actions
Capture Team
Capture Team
Board actions
Board actions
Enterprise Stability Issues 🤕
Enterprise Stability Issues 🤕
Board actions
Board actions
Processing Team
Processing Team
Board actions
Board actions
SE Kanban
SE Kanban
Board actions
Board actions
Service-Desk
Service-Desk
More actions for Service-Desk...
|
65222
|
NULL
|
NULL
|
NULL
|
|
65228
|
2286
|
22
|
2026-05-20T15:29:47.526819+00:00
|
/Users/lukas/.screenpipe/data/data/2026-05-20/1779 /Users/lukas/.screenpipe/data/data/2026-05-20/1779290987526_m2.jpg...
|
Firefox
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira — Work...
|
1
|
jiminny.atlassian.net/jira/software/c/projects/JY/ jiminny.atlassian.net/jira/software/c/projects/JY/boards/37...
|
monitor_2
|
NULL
|
NULL
|
NULL
|
NULL
|
Allow owner's role to be selected when setting Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
github.com
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Close tab
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Close tab
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics....
|
[{"role":"AXStaticText","text& [{"role":"AXStaticText","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":4,"bounds":{"left":0.08361037,"top":0.15722266,"width":0.08460771,"height":0.032322425},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"github.com","depth":4,"bounds":{"left":0.08361037,"top":0.17877094,"width":0.019614361,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":4,"bounds":{"left":0.0,"top":0.0518755,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.06304868,"width":0.10106383,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Close tab","depth":5,"bounds":{"left":0.06732048,"top":0.05905826,"width":0.007978723,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXRadioButton","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":4,"bounds":{"left":0.0,"top":0.08459697,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.09577015,"width":0.10721409,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.11731844,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.12849163,"width":0.15791224,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.15003991,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.16121309,"width":0.1796875,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Close tab","depth":5,"bounds":{"left":0.06732048,"top":0.15722266,"width":0.007978723,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXRadioButton","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.18276137,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.19393456,"width":0.12699468,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":4,"bounds":{"left":0.0,"top":0.21548285,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":5,"bounds":{"left":0.013297873,"top":0.22665602,"width":0.10139628,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":4,"bounds":{"left":0.0,"top":0.2482043,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.25937748,"width":0.18068483,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":4,"bounds":{"left":0.0,"top":0.28092578,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":5,"bounds":{"left":0.013297873,"top":0.29209897,"width":0.4644282,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Pipelines - jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.31364724,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Pipelines - jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.32482043,"width":0.039228722,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.3463687,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.3575419,"width":0.15824468,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Tab","depth":4,"bounds":{"left":0.0028257978,"top":0.38068634,"width":0.07413564,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Customize sidebar","depth":6,"bounds":{"left":0.0028257978,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Close Google Gemini (⌃X)","depth":6,"bounds":{"left":0.013796543,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Tabs from other devices","depth":6,"bounds":{"left":0.024933511,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open history (⇧⌘H)","depth":6,"bounds":{"left":0.036070477,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open bookmarks (⌘B)","depth":6,"bounds":{"left":0.04720745,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"AI Chat settings","depth":7,"bounds":{"left":0.35854387,"top":0.055067837,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Close","depth":7,"bounds":{"left":0.37051198,"top":0.055067837,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Main menu","depth":13,"bounds":{"left":0.08228058,"top":0.09736632,"width":0.015957447,"height":0.03830806},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"New chat","depth":12,"bounds":{"left":0.09823803,"top":0.10215483,"width":0.028091755,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Gemini","depth":15,"bounds":{"left":0.1008976,"top":0.10654429,"width":0.02144282,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Chat","depth":13,"bounds":{"left":0.35854387,"top":0.10215483,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Open menu for conversation actions.","depth":13,"bounds":{"left":0.37051198,"top":0.10215483,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Conversation with Gemini","depth":15,"bounds":{"left":0.079288565,"top":0.14445332,"width":0.0003324468,"height":0.0007980846},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Conversation with Gemini","depth":16,"bounds":{"left":0.079288565,"top":0.14684756,"width":0.1200133,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"You said I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.","depth":23,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Expand","depth":21,"on_screen":false,"role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a concise summary of the changes made in the code selection:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Refactoring and Dependencies","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Refactoring and Dependencies","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Replaced Interface with Implementation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Changed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusInterface","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusManager","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"in the imports and constructor dependency injection.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Redis Support:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Imported","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Illuminate\\Support\\Facades\\Redis","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to handle a newly introduced memory-logging lock mechanism.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Traits:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"WorkerIdTrait","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", along with the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"worker","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"field tracking in the final logs.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Loop and Worker Lifecycle Adjustments","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Loop and Worker Lifecycle Adjustments","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Worker Auto-Restart:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Deleted the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property, the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"constant, and the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"shouldIterateAgain()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method. The script no longer tracks execution iterations or triggers a self-managed worker restart.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Simplified Chunk Method Return:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Modified","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to always return","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Priority Handling for Throttled Entities","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Priority Handling for Throttled Entities","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Stale Record Elevation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Updated the logic in","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"rescheduleThrottledEntities()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to check if a throttled entity group is classified as \"stale\". If it is, the entities are rescheduled using","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"QueuePriorityEnum::NORMAL","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"; otherwise, they keep their respective queue priority.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Metrics, Logging, and Environment Handling","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Metrics, Logging, and Environment Handling","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Environment Tracking:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Introduced a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$environment","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property (defaulting to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"'local'","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") and a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"setEnvironment()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method to dynamically assign it.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Memory Monitoring via Datadog Gauge:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method that submits peak memory allocation to Datadog as a gauge metric (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"jiminny.reindex-memory-usage","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis Lock for Metrics Throttling:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Implemented a Redis-backed lock (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"%s-process-manager-lock","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") with a 60-second expiration inside","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to prevent over-reporting metrics.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"}]...
|
-8984720678585540338
|
-1025845918813554724
|
click
|
accessibility
|
NULL
|
Allow owner's role to be selected when setting Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
github.com
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Close tab
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Close tab
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics....
|
NULL
|
NULL
|
NULL
|
NULL
|
|
65229
|
2286
|
23
|
2026-05-20T15:29:52.628846+00:00
|
/Users/lukas/.screenpipe/data/data/2026-05-20/1779 /Users/lukas/.screenpipe/data/data/2026-05-20/1779290992628_m2.jpg...
|
Firefox
|
JY-20676 delete AJ reports related objects by Laky JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app — Work...
|
1
|
github.com/jiminny/app/pull/12098
|
monitor_2
|
NULL
|
NULL
|
NULL
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Close tab
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud...
|
[{"role":"AXRadioButton","text [{"role":"AXRadioButton","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":4,"bounds":{"left":0.0,"top":0.0518755,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.06304868,"width":0.10106383,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":4,"bounds":{"left":0.0,"top":0.08459697,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.09577015,"width":0.10721409,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.11731844,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.12849163,"width":0.15791224,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Close tab","depth":5,"bounds":{"left":0.06732048,"top":0.1245012,"width":0.007978723,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXRadioButton","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.15003991,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.16121309,"width":0.1796875,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.18276137,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.19393456,"width":0.12699468,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":4,"bounds":{"left":0.0,"top":0.21548285,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":5,"bounds":{"left":0.013297873,"top":0.22665602,"width":0.10139628,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"}]...
|
-6920300281940536902
|
-1130553543926976055
|
click
|
accessibility
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Close tab
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud...
|
NULL
|
NULL
|
NULL
|
NULL
|
|
65230
|
2286
|
24
|
2026-05-20T15:29:54.584269+00:00
|
/Users/lukas/.screenpipe/data/data/2026-05-20/1779 /Users/lukas/.screenpipe/data/data/2026-05-20/1779290994584_m2.jpg...
|
Firefox
|
JY-20676 delete AJ reports related objects by Laky JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app — Work...
|
1
|
github.com/jiminny/app/pull/12098
|
monitor_2
|
NULL
|
NULL
|
NULL
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Close tab
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo
Share & export
Copy
Show more options
Ask Gemini
Ask Gemini
Upload & tools
Microphone
Send message
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Summarize page
Summarize page
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
All issues(g then i)
All pull requests
All repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Pull requests (31)
Pull requests
(
31
)
Agents
Agents
Actions
Actions
Wiki
Wiki
Security and quality
Security and quality
Insights
Insights
Settings
Settings
JY-20676 delete AJ reports related objects #12098 Edit title
JY-20676 delete AJ reports related objects
#
12098
Edit title
Ready to merge
Ready to merge
Code
Code
Open
LakyLak
LakyLak
wants to merge 21 commits into
master
master
from
JY-20676-delete-report-related-objects
JY-20676-delete-report-related-objects
Copy head branch name to clipboard
Lines changed: 1698 additions & 1471 deletions...
|
[{"role":"AXRadioButton","text [{"role":"AXRadioButton","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":4,"bounds":{"left":0.0,"top":0.0518755,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.06304868,"width":0.10106383,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":4,"bounds":{"left":0.0,"top":0.08459697,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.09577015,"width":0.10721409,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.11731844,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.12849163,"width":0.15791224,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Close tab","depth":5,"bounds":{"left":0.06732048,"top":0.1245012,"width":0.007978723,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXRadioButton","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.15003991,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.16121309,"width":0.1796875,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.18276137,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.19393456,"width":0.12699468,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Close tab","depth":5,"bounds":{"left":0.06732048,"top":0.18994413,"width":0.007978723,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXRadioButton","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":4,"bounds":{"left":0.0,"top":0.21548285,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":5,"bounds":{"left":0.013297873,"top":0.22665602,"width":0.10139628,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":4,"bounds":{"left":0.0,"top":0.2482043,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.25937748,"width":0.18068483,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":4,"bounds":{"left":0.0,"top":0.28092578,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":5,"bounds":{"left":0.013297873,"top":0.29209897,"width":0.4644282,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Pipelines - jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.31364724,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Pipelines - jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.32482043,"width":0.039228722,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.3463687,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.3575419,"width":0.15824468,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Tab","depth":4,"bounds":{"left":0.0028257978,"top":0.38068634,"width":0.07413564,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Customize sidebar","depth":6,"bounds":{"left":0.0028257978,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Close Google Gemini (⌃X)","depth":6,"bounds":{"left":0.013796543,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Tabs from other devices","depth":6,"bounds":{"left":0.024933511,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open history (⇧⌘H)","depth":6,"bounds":{"left":0.036070477,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open bookmarks (⌘B)","depth":6,"bounds":{"left":0.04720745,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"AI Chat settings","depth":7,"bounds":{"left":0.35854387,"top":0.055067837,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Close","depth":7,"bounds":{"left":0.37051198,"top":0.055067837,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Main menu","depth":13,"bounds":{"left":0.08228058,"top":0.09736632,"width":0.015957447,"height":0.03830806},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"New chat","depth":12,"bounds":{"left":0.09823803,"top":0.10215483,"width":0.028091755,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Gemini","depth":15,"bounds":{"left":0.1008976,"top":0.10654429,"width":0.02144282,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Chat","depth":13,"bounds":{"left":0.35854387,"top":0.10215483,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Open menu for conversation actions.","depth":13,"bounds":{"left":0.37051198,"top":0.10215483,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Conversation with Gemini","depth":15,"bounds":{"left":0.079288565,"top":0.14445332,"width":0.0003324468,"height":0.0007980846},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Conversation with Gemini","depth":16,"bounds":{"left":0.079288565,"top":0.14684756,"width":0.1200133,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"You said I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.","depth":23,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Expand","depth":21,"on_screen":false,"role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a concise summary of the changes made in the code selection:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Refactoring and Dependencies","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Refactoring and Dependencies","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Replaced Interface with Implementation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Changed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusInterface","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusManager","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"in the imports and constructor dependency injection.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Redis Support:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Imported","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Illuminate\\Support\\Facades\\Redis","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to handle a newly introduced memory-logging lock mechanism.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Traits:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"WorkerIdTrait","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", along with the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"worker","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"field tracking in the final logs.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Loop and Worker Lifecycle Adjustments","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Loop and Worker Lifecycle Adjustments","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Worker Auto-Restart:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Deleted the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property, the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"constant, and the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"shouldIterateAgain()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method. The script no longer tracks execution iterations or triggers a self-managed worker restart.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Simplified Chunk Method Return:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Modified","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to always return","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Priority Handling for Throttled Entities","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Priority Handling for Throttled Entities","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Stale Record Elevation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Updated the logic in","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"rescheduleThrottledEntities()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to check if a throttled entity group is classified as \"stale\". If it is, the entities are rescheduled using","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"QueuePriorityEnum::NORMAL","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"; otherwise, they keep their respective queue priority.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Metrics, Logging, and Environment Handling","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Metrics, Logging, and Environment Handling","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Environment Tracking:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Introduced a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$environment","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property (defaulting to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"'local'","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") and a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"setEnvironment()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method to dynamically assign it.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Memory Monitoring via Datadog Gauge:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method that submits peak memory allocation to Datadog as a gauge metric (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"jiminny.reindex-memory-usage","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis Lock for Metrics Throttling:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Implemented a Redis-backed lock (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"%s-process-manager-lock","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") with a 60-second expiration inside","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to prevent over-reporting metrics.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Log Payload Update:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added total processed items (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$updatedDocuments + $deletedDocuments","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") to the final information log.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":23,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"You said are there any issues or potential bottlenecks","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"are there any issues or potential bottlenecks","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Edit","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a breakdown of the potential issues to look out for:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"1. Memory Leak Risk (OOM Crashes)","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"1. Memory Leak Risk (OOM Crashes)","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Restart Logic:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The most significant change is the removal of the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Dependency Leaks:","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.050033245,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If any underlying service (like the","depth":26,"bounds":{"left":0.17453457,"top":0.0,"width":0.08277926,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ElasticSearchClient","depth":27,"bounds":{"left":0.25930852,"top":0.0,"width":0.056848403,"height":0.015163607},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", ORM, or Datadog helper) retains references to processed entities,","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.21775267,"height":0.03631285},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"bounds":{"left":0.26811835,"top":0.0,"width":0.056848403,"height":0.015163607},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.23138298,"height":0.05546688},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Hardcoded Limits:","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.047041222,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The code still includes","depth":26,"bounds":{"left":0.17154256,"top":0.0,"width":0.05651596,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ini_set('memory_limit', '512M');","depth":27,"bounds":{"left":0.23005319,"top":0.0,"width":0.095744684,"height":0.015163607},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":". Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.23055187,"height":0.05546688},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"2. Race Condition in Redis Lock","depth":23,"bounds":{"left":0.111369684,"top":0.07940942,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"2. Race Condition in Redis Lock","depth":24,"bounds":{"left":0.111369684,"top":0.079010375,"width":0.09125665,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Non-Atomic Operations:","depth":26,"bounds":{"left":0.12450133,"top":0.10614525,"width":0.062333778,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The","depth":26,"bounds":{"left":0.18683511,"top":0.10614525,"width":0.011635638,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage","depth":27,"bounds":{"left":0.20046543,"top":0.10774142,"width":0.041888297,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method uses three distinct steps to handle the lock:","depth":26,"bounds":{"left":0.12450133,"top":0.10614525,"width":0.22722739,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"get()","depth":27,"bounds":{"left":0.14860372,"top":0.12689546,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":",","depth":26,"bounds":{"left":0.16555852,"top":0.12529927,"width":0.002493351,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"bounds":{"left":0.17004654,"top":0.12689546,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", and","depth":26,"bounds":{"left":0.18700133,"top":0.12529927,"width":0.013131649,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"expire()","depth":27,"bounds":{"left":0.20212767,"top":0.12689546,"width":0.023936171,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"bounds":{"left":0.22805852,"top":0.12529927,"width":0.0013297872,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"bounds":{"left":0.12450133,"top":0.15403032,"width":0.040059842,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If multiple workers process the same","depth":26,"bounds":{"left":0.16456117,"top":0.15403032,"width":0.09275266,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$this->updateTarget","depth":27,"bounds":{"left":0.25930852,"top":0.15562649,"width":0.056848403,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"concurrently, they can both pass the","depth":26,"bounds":{"left":0.12450133,"top":0.15403032,"width":0.22855718,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"if (! is_null(Redis::get(...)))","depth":27,"bounds":{"left":0.18351063,"top":0.17478053,"width":0.09275266,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"check at the exact same millisecond. Both will then execute","depth":26,"bounds":{"left":0.12450133,"top":0.17318435,"width":0.21575798,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"bounds":{"left":0.2122673,"top":0.19393456,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and push duplicate metrics to Datadog.","depth":26,"bounds":{"left":0.22922207,"top":0.19233839,"width":0.09823803,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Fix:","depth":26,"bounds":{"left":0.12450133,"top":0.22106944,"width":0.019281914,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"This should be replaced with a single atomic operation using Redis's","depth":26,"bounds":{"left":0.14378324,"top":0.22106944,"width":0.16821809,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"NX","depth":27,"bounds":{"left":0.31399602,"top":0.22266561,"width":0.005984043,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Not eXists) and","depth":26,"bounds":{"left":0.12450133,"top":0.22106944,"width":0.22855718,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"EX","depth":27,"bounds":{"left":0.13696809,"top":0.24181964,"width":0.005984043,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Expire) flags, or Laravel’s built-in atomic cache locks:","depth":26,"bounds":{"left":0.14494681,"top":0.24022347,"width":0.13264628,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis::set($gaugeLockName, true, 'EX', 60, 'NX')","depth":27,"bounds":{"left":0.12649602,"top":0.26097366,"width":0.14361702,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"bounds":{"left":0.27210772,"top":0.25937748,"width":0.0013297872,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"3. Infinite Loops and Caller Contract","depth":23,"bounds":{"left":0.111369684,"top":0.30925778,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"3. Infinite Loops and Caller Contract","depth":24,"bounds":{"left":0.111369684,"top":0.30885875,"width":0.10571808,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Return Value Change:","depth":26,"bounds":{"left":0.12450133,"top":0.33599362,"width":0.055352394,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"bounds":{"left":0.18301196,"top":0.33758977,"width":0.06881649,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"previously returned whether it should iterate again based on staleness or chunk size. It now always returns","depth":26,"bounds":{"left":0.12450133,"top":0.33599362,"width":0.2237367,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"bounds":{"left":0.29438165,"top":0.3567438,"width":0.011968086,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(unless the selection is totally empty).","depth":26,"bounds":{"left":0.12450133,"top":0.35514766,"width":0.2130984,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"bounds":{"left":0.12450133,"top":0.40303272,"width":0.040059842,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the code calling this method relies on a","depth":26,"bounds":{"left":0.16456117,"top":0.40303272,"width":0.10305851,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"false","depth":27,"bounds":{"left":0.26961437,"top":0.4046289,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"return to pause, sleep, or gracefully terminate, changing this to an unconditional","depth":26,"bounds":{"left":0.12450133,"top":0.40303272,"width":0.22589761,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"bounds":{"left":0.26163563,"top":0.4237829,"width":0.011968086,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"could result in an aggressive \"busy loop\" that spikes CPU usage.","depth":26,"bounds":{"left":0.12450133,"top":0.42218676,"width":0.22290559,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"4. Performance Overhead","depth":23,"bounds":{"left":0.111369684,"top":0.49122107,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"4. Performance Overhead","depth":24,"bounds":{"left":0.111369684,"top":0.49082202,"width":0.076130316,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Expensive Garbage Collection:","depth":26,"bounds":{"left":0.12450133,"top":0.5179569,"width":0.078457445,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Calling","depth":26,"bounds":{"left":0.20295878,"top":0.5179569,"width":0.019281914,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles();","depth":27,"bounds":{"left":0.22423537,"top":0.51955307,"width":0.059840426,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"on","depth":26,"bounds":{"left":0.28607047,"top":0.5179569,"width":0.00880984,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"every single chunk","depth":26,"bounds":{"left":0.29488033,"top":0.5179569,"width":0.045545213,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.","depth":26,"bounds":{"left":0.12450133,"top":0.5179569,"width":0.23420878,"height":0.05546688},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Synchronous Metric Dispatch:","depth":26,"bounds":{"left":0.12450133,"top":0.584996,"width":0.07712766,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the","depth":26,"bounds":{"left":0.20162898,"top":0.584996,"width":0.015292553,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::increment","depth":27,"bounds":{"left":0.21891622,"top":0.5865922,"width":0.053856384,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"or","depth":26,"bounds":{"left":0.27476728,"top":0.584996,"width":0.0078125,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::gauge","depth":27,"bounds":{"left":0.28457448,"top":0.5865922,"width":0.041888297,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.","depth":26,"bounds":{"left":0.12450133,"top":0.584996,"width":0.23420878,"height":0.05546688},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"5. Lack of Explicit Error Handling","depth":23,"bounds":{"left":0.111369684,"top":0.67318434,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5. Lack of Explicit Error Handling","depth":24,"bounds":{"left":0.111369684,"top":0.67278534,"width":0.094913565,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"No","depth":26,"bounds":{"left":0.12450133,"top":0.6999202,"width":0.008477394,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"try/catch","depth":27,"bounds":{"left":0.1349734,"top":0.7015164,"width":0.026928192,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Blocks:","depth":26,"bounds":{"left":0.16389628,"top":0.6999202,"width":0.019614361,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The script executes complex network operations (","depth":26,"bounds":{"left":0.18351063,"top":0.6999202,"width":0.12367021,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"loadDocuments","depth":27,"bounds":{"left":0.30917552,"top":0.7015164,"width":0.038896278,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.","depth":26,"bounds":{"left":0.12450133,"top":0.6999202,"width":0.22888963,"height":0.09377494},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":22,"bounds":{"left":0.1087101,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":22,"bounds":{"left":0.124667555,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Redo","depth":22,"bounds":{"left":0.140625,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":21,"bounds":{"left":0.15658244,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":22,"bounds":{"left":0.17253989,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":21,"bounds":{"left":0.18849733,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXTextArea","text":"Ask Gemini","depth":21,"bounds":{"left":0.14594415,"top":0.86632085,"width":0.17021276,"height":0.01915403},"on_screen":true,"value":"Ask Gemini","help_text":"","role_description":"text entry area","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Ask Gemini","depth":22,"bounds":{"left":0.14594415,"top":0.8667199,"width":0.030418882,"height":0.018355945},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Upload & tools","depth":20,"bounds":{"left":0.12732713,"top":0.8567438,"width":0.015957447,"height":0.03830806},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Microphone","depth":20,"bounds":{"left":0.32413563,"top":0.86153233,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Send message","depth":20,"bounds":{"left":0.33211437,"top":0.8631285,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":false,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.","depth":17,"bounds":{"left":0.102726065,"top":0.9173983,"width":0.21775267,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Your privacy & Gemini Opens in a new window","depth":17,"bounds":{"left":0.32047874,"top":0.9173983,"width":0.043882977,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Your privacy & Gemini","depth":18,"bounds":{"left":0.32047874,"top":0.9173983,"width":0.043882977,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Opens in a new window","depth":19,"bounds":{"left":0.079288565,"top":0.91660017,"width":0.04720745,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Summarize page","depth":7,"bounds":{"left":0.08494016,"top":0.95730245,"width":0.053523935,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Summarize page","depth":9,"bounds":{"left":0.09059176,"top":0.96249,"width":0.042220745,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Skip to content","depth":6,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Skip to content","depth":7,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Open menu","depth":10,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"Homepage (g then d)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"jiminny","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"jiminny","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"app","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"app","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Search or jump to…","depth":9,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Type","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"/","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to search","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Chat with Copilot","depth":10,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXMenuButton","text":"Open Copilot…","depth":9,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXMenuButton","text":"Create new...","depth":9,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"All issues(g then i)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"All pull requests","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"All repositories","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"You have unread notifications(g then n)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Open user navigation menu","depth":9,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Repository navigation","depth":9,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Repository navigation","depth":10,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Code","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Code","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Pull requests (31)","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Pull requests","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"31","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Agents","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Agents","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Actions","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Actions","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Wiki","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Wiki","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Security and quality","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Security and quality","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Insights","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Insights","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Settings","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Settings","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"JY-20676 delete AJ reports related objects #12098 Edit title","depth":13,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"JY-20676 delete AJ reports related objects","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"#","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"12098","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Edit title","depth":14,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Ready to merge","depth":13,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Ready to merge","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXMenuButton","text":"Code","depth":13,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXStaticText","text":"Code","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Open","depth":13,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"LakyLak","depth":15,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"wants to merge 21 commits into","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"master","depth":15,"on_screen":false,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"master","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"from","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20676-delete-report-related-objects","depth":16,"on_screen":false,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20676-delete-report-related-objects","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy head branch name to clipboard","depth":16,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Lines changed: 1698 additions & 1471 deletions","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"}]...
|
-7546986204830668845
|
-1052796009671461092
|
visual_change
|
accessibility
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Close tab
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo
Share & export
Copy
Show more options
Ask Gemini
Ask Gemini
Upload & tools
Microphone
Send message
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Summarize page
Summarize page
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
All issues(g then i)
All pull requests
All repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Pull requests (31)
Pull requests
(
31
)
Agents
Agents
Actions
Actions
Wiki
Wiki
Security and quality
Security and quality
Insights
Insights
Settings
Settings
JY-20676 delete AJ reports related objects #12098 Edit title
JY-20676 delete AJ reports related objects
#
12098
Edit title
Ready to merge
Ready to merge
Code
Code
Open
LakyLak
LakyLak
wants to merge 21 commits into
master
master
from
JY-20676-delete-report-related-objects
JY-20676-delete-report-related-objects
Copy head branch name to clipboard
Lines changed: 1698 additions & 1471 deletions...
|
65229
|
NULL
|
NULL
|
NULL
|
|
65231
|
2286
|
25
|
2026-05-20T15:29:55.322131+00:00
|
/Users/lukas/.screenpipe/data/data/2026-05-20/1779 /Users/lukas/.screenpipe/data/data/2026-05-20/1779290995322_m2.jpg...
|
Firefox
|
JY-20915 add alias for EU by LakyLak · Pull Reques JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app — Work...
|
1
|
github.com/jiminny/app/pull/12105
|
monitor_2
|
NULL
|
NULL
|
NULL
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo...
|
[{"role":"AXRadioButton","text [{"role":"AXRadioButton","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":4,"bounds":{"left":0.0,"top":0.0518755,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.06304868,"width":0.10106383,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":4,"bounds":{"left":0.0,"top":0.08459697,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.09577015,"width":0.10721409,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.11731844,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.12849163,"width":0.15791224,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.15003991,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.16121309,"width":0.1796875,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.18276137,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.19393456,"width":0.12699468,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Close tab","depth":5,"bounds":{"left":0.06732048,"top":0.18994413,"width":0.007978723,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXRadioButton","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":4,"bounds":{"left":0.0,"top":0.21548285,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":5,"bounds":{"left":0.013297873,"top":0.22665602,"width":0.10139628,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":4,"bounds":{"left":0.0,"top":0.2482043,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.25937748,"width":0.18068483,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":4,"bounds":{"left":0.0,"top":0.28092578,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":5,"bounds":{"left":0.013297873,"top":0.29209897,"width":0.4644282,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Pipelines - jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.31364724,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Pipelines - jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.32482043,"width":0.039228722,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.3463687,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.3575419,"width":0.15824468,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Tab","depth":4,"bounds":{"left":0.0028257978,"top":0.38068634,"width":0.07413564,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Customize sidebar","depth":6,"bounds":{"left":0.0028257978,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Close Google Gemini (⌃X)","depth":6,"bounds":{"left":0.013796543,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Tabs from other devices","depth":6,"bounds":{"left":0.024933511,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open history (⇧⌘H)","depth":6,"bounds":{"left":0.036070477,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open bookmarks (⌘B)","depth":6,"bounds":{"left":0.04720745,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"AI Chat settings","depth":7,"bounds":{"left":0.35854387,"top":0.055067837,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Close","depth":7,"bounds":{"left":0.37051198,"top":0.055067837,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Main menu","depth":13,"bounds":{"left":0.08228058,"top":0.09736632,"width":0.015957447,"height":0.03830806},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"New chat","depth":12,"bounds":{"left":0.09823803,"top":0.10215483,"width":0.028091755,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Gemini","depth":15,"bounds":{"left":0.1008976,"top":0.10654429,"width":0.02144282,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Chat","depth":13,"bounds":{"left":0.35854387,"top":0.10215483,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Open menu for conversation actions.","depth":13,"bounds":{"left":0.37051198,"top":0.10215483,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Conversation with Gemini","depth":15,"bounds":{"left":0.079288565,"top":0.14445332,"width":0.0003324468,"height":0.0007980846},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Conversation with Gemini","depth":16,"bounds":{"left":0.079288565,"top":0.14684756,"width":0.1200133,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"You said I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.","depth":23,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Expand","depth":21,"on_screen":false,"role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a concise summary of the changes made in the code selection:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Refactoring and Dependencies","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Refactoring and Dependencies","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Replaced Interface with Implementation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Changed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusInterface","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusManager","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"in the imports and constructor dependency injection.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Redis Support:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Imported","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Illuminate\\Support\\Facades\\Redis","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to handle a newly introduced memory-logging lock mechanism.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Traits:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"WorkerIdTrait","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", along with the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"worker","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"field tracking in the final logs.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Loop and Worker Lifecycle Adjustments","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Loop and Worker Lifecycle Adjustments","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Worker Auto-Restart:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Deleted the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property, the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"constant, and the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"shouldIterateAgain()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method. The script no longer tracks execution iterations or triggers a self-managed worker restart.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Simplified Chunk Method Return:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Modified","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to always return","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Priority Handling for Throttled Entities","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Priority Handling for Throttled Entities","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Stale Record Elevation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Updated the logic in","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"rescheduleThrottledEntities()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to check if a throttled entity group is classified as \"stale\". If it is, the entities are rescheduled using","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"QueuePriorityEnum::NORMAL","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"; otherwise, they keep their respective queue priority.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Metrics, Logging, and Environment Handling","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Metrics, Logging, and Environment Handling","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Environment Tracking:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Introduced a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$environment","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property (defaulting to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"'local'","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") and a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"setEnvironment()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method to dynamically assign it.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Memory Monitoring via Datadog Gauge:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method that submits peak memory allocation to Datadog as a gauge metric (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"jiminny.reindex-memory-usage","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis Lock for Metrics Throttling:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Implemented a Redis-backed lock (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"%s-process-manager-lock","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") with a 60-second expiration inside","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to prevent over-reporting metrics.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Log Payload Update:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added total processed items (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$updatedDocuments + $deletedDocuments","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") to the final information log.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":23,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"You said are there any issues or potential bottlenecks","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"are there any issues or potential bottlenecks","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Edit","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a breakdown of the potential issues to look out for:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"1. Memory Leak Risk (OOM Crashes)","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"1. Memory Leak Risk (OOM Crashes)","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Restart Logic:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The most significant change is the removal of the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Dependency Leaks:","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.050033245,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If any underlying service (like the","depth":26,"bounds":{"left":0.17453457,"top":0.0,"width":0.08277926,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ElasticSearchClient","depth":27,"bounds":{"left":0.25930852,"top":0.0,"width":0.056848403,"height":0.015163607},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", ORM, or Datadog helper) retains references to processed entities,","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.21775267,"height":0.03631285},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"bounds":{"left":0.26811835,"top":0.0,"width":0.056848403,"height":0.015163607},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.23138298,"height":0.05546688},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Hardcoded Limits:","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.047041222,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The code still includes","depth":26,"bounds":{"left":0.17154256,"top":0.0,"width":0.05651596,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ini_set('memory_limit', '512M');","depth":27,"bounds":{"left":0.23005319,"top":0.0,"width":0.095744684,"height":0.015163607},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":". Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.23055187,"height":0.05546688},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"2. Race Condition in Redis Lock","depth":23,"bounds":{"left":0.111369684,"top":0.07940942,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"2. Race Condition in Redis Lock","depth":24,"bounds":{"left":0.111369684,"top":0.079010375,"width":0.09125665,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Non-Atomic Operations:","depth":26,"bounds":{"left":0.12450133,"top":0.10614525,"width":0.062333778,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The","depth":26,"bounds":{"left":0.18683511,"top":0.10614525,"width":0.011635638,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage","depth":27,"bounds":{"left":0.20046543,"top":0.10774142,"width":0.041888297,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method uses three distinct steps to handle the lock:","depth":26,"bounds":{"left":0.12450133,"top":0.10614525,"width":0.22722739,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"get()","depth":27,"bounds":{"left":0.14860372,"top":0.12689546,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":",","depth":26,"bounds":{"left":0.16555852,"top":0.12529927,"width":0.002493351,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"bounds":{"left":0.17004654,"top":0.12689546,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", and","depth":26,"bounds":{"left":0.18700133,"top":0.12529927,"width":0.013131649,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"expire()","depth":27,"bounds":{"left":0.20212767,"top":0.12689546,"width":0.023936171,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"bounds":{"left":0.22805852,"top":0.12529927,"width":0.0013297872,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"bounds":{"left":0.12450133,"top":0.15403032,"width":0.040059842,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If multiple workers process the same","depth":26,"bounds":{"left":0.16456117,"top":0.15403032,"width":0.09275266,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$this->updateTarget","depth":27,"bounds":{"left":0.25930852,"top":0.15562649,"width":0.056848403,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"concurrently, they can both pass the","depth":26,"bounds":{"left":0.12450133,"top":0.15403032,"width":0.22855718,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"if (! is_null(Redis::get(...)))","depth":27,"bounds":{"left":0.18351063,"top":0.17478053,"width":0.09275266,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"check at the exact same millisecond. Both will then execute","depth":26,"bounds":{"left":0.12450133,"top":0.17318435,"width":0.21575798,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"bounds":{"left":0.2122673,"top":0.19393456,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and push duplicate metrics to Datadog.","depth":26,"bounds":{"left":0.22922207,"top":0.19233839,"width":0.09823803,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Fix:","depth":26,"bounds":{"left":0.12450133,"top":0.22106944,"width":0.019281914,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"This should be replaced with a single atomic operation using Redis's","depth":26,"bounds":{"left":0.14378324,"top":0.22106944,"width":0.16821809,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"NX","depth":27,"bounds":{"left":0.31399602,"top":0.22266561,"width":0.005984043,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Not eXists) and","depth":26,"bounds":{"left":0.12450133,"top":0.22106944,"width":0.22855718,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"EX","depth":27,"bounds":{"left":0.13696809,"top":0.24181964,"width":0.005984043,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Expire) flags, or Laravel’s built-in atomic cache locks:","depth":26,"bounds":{"left":0.14494681,"top":0.24022347,"width":0.13264628,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis::set($gaugeLockName, true, 'EX', 60, 'NX')","depth":27,"bounds":{"left":0.12649602,"top":0.26097366,"width":0.14361702,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"bounds":{"left":0.27210772,"top":0.25937748,"width":0.0013297872,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"3. Infinite Loops and Caller Contract","depth":23,"bounds":{"left":0.111369684,"top":0.30925778,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"3. Infinite Loops and Caller Contract","depth":24,"bounds":{"left":0.111369684,"top":0.30885875,"width":0.10571808,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Return Value Change:","depth":26,"bounds":{"left":0.12450133,"top":0.33599362,"width":0.055352394,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"bounds":{"left":0.18301196,"top":0.33758977,"width":0.06881649,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"previously returned whether it should iterate again based on staleness or chunk size. It now always returns","depth":26,"bounds":{"left":0.12450133,"top":0.33599362,"width":0.2237367,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"bounds":{"left":0.29438165,"top":0.3567438,"width":0.011968086,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(unless the selection is totally empty).","depth":26,"bounds":{"left":0.12450133,"top":0.35514766,"width":0.2130984,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"bounds":{"left":0.12450133,"top":0.40303272,"width":0.040059842,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the code calling this method relies on a","depth":26,"bounds":{"left":0.16456117,"top":0.40303272,"width":0.10305851,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"false","depth":27,"bounds":{"left":0.26961437,"top":0.4046289,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"return to pause, sleep, or gracefully terminate, changing this to an unconditional","depth":26,"bounds":{"left":0.12450133,"top":0.40303272,"width":0.22589761,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"bounds":{"left":0.26163563,"top":0.4237829,"width":0.011968086,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"could result in an aggressive \"busy loop\" that spikes CPU usage.","depth":26,"bounds":{"left":0.12450133,"top":0.42218676,"width":0.22290559,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"4. Performance Overhead","depth":23,"bounds":{"left":0.111369684,"top":0.49122107,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"4. Performance Overhead","depth":24,"bounds":{"left":0.111369684,"top":0.49082202,"width":0.076130316,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Expensive Garbage Collection:","depth":26,"bounds":{"left":0.12450133,"top":0.5179569,"width":0.078457445,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Calling","depth":26,"bounds":{"left":0.20295878,"top":0.5179569,"width":0.019281914,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles();","depth":27,"bounds":{"left":0.22423537,"top":0.51955307,"width":0.059840426,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"on","depth":26,"bounds":{"left":0.28607047,"top":0.5179569,"width":0.00880984,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"every single chunk","depth":26,"bounds":{"left":0.29488033,"top":0.5179569,"width":0.045545213,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.","depth":26,"bounds":{"left":0.12450133,"top":0.5179569,"width":0.23420878,"height":0.05546688},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Synchronous Metric Dispatch:","depth":26,"bounds":{"left":0.12450133,"top":0.584996,"width":0.07712766,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the","depth":26,"bounds":{"left":0.20162898,"top":0.584996,"width":0.015292553,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::increment","depth":27,"bounds":{"left":0.21891622,"top":0.5865922,"width":0.053856384,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"or","depth":26,"bounds":{"left":0.27476728,"top":0.584996,"width":0.0078125,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::gauge","depth":27,"bounds":{"left":0.28457448,"top":0.5865922,"width":0.041888297,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.","depth":26,"bounds":{"left":0.12450133,"top":0.584996,"width":0.23420878,"height":0.05546688},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"5. Lack of Explicit Error Handling","depth":23,"bounds":{"left":0.111369684,"top":0.67318434,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5. Lack of Explicit Error Handling","depth":24,"bounds":{"left":0.111369684,"top":0.67278534,"width":0.094913565,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"No","depth":26,"bounds":{"left":0.12450133,"top":0.6999202,"width":0.008477394,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"try/catch","depth":27,"bounds":{"left":0.1349734,"top":0.7015164,"width":0.026928192,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Blocks:","depth":26,"bounds":{"left":0.16389628,"top":0.6999202,"width":0.019614361,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The script executes complex network operations (","depth":26,"bounds":{"left":0.18351063,"top":0.6999202,"width":0.12367021,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"loadDocuments","depth":27,"bounds":{"left":0.30917552,"top":0.7015164,"width":0.038896278,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.","depth":26,"bounds":{"left":0.12450133,"top":0.6999202,"width":0.22888963,"height":0.09377494},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":22,"bounds":{"left":0.1087101,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":22,"bounds":{"left":0.124667555,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Redo","depth":22,"bounds":{"left":0.140625,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false}]...
|
8583392855393166325
|
-1052937881027004452
|
click
|
accessibility
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo...
|
NULL
|
NULL
|
NULL
|
NULL
|
|
65232
|
NULL
|
0
|
2026-05-20T15:29:57.589865+00:00
|
/Users/lukas/.screenpipe/data/data/2026-05-20/1779 /Users/lukas/.screenpipe/data/data/2026-05-20/1779290997589_m2.jpg...
|
Firefox
|
JY-20915 add alias for EU by LakyLak · Pull Reques JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app — Work...
|
1
|
github.com/jiminny/app/pull/12105
|
monitor_2
|
NULL
|
NULL
|
NULL
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo
Share & export
Copy
Show more options
Ask Gemini
Ask Gemini
Upload & tools
Microphone
Send message
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Summarize page
Summarize page
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app...
|
[{"role":"AXRadioButton","text [{"role":"AXRadioButton","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":4,"bounds":{"left":0.0,"top":0.0518755,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.06304868,"width":0.10106383,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":4,"bounds":{"left":0.0,"top":0.08459697,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.09577015,"width":0.10721409,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.11731844,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.12849163,"width":0.15791224,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.15003991,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.16121309,"width":0.1796875,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.18276137,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.19393456,"width":0.12699468,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Close tab","depth":5,"bounds":{"left":0.06732048,"top":0.18994413,"width":0.007978723,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXRadioButton","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":4,"bounds":{"left":0.0,"top":0.21548285,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":5,"bounds":{"left":0.013297873,"top":0.22665602,"width":0.10139628,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":4,"bounds":{"left":0.0,"top":0.2482043,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.25937748,"width":0.18068483,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":4,"bounds":{"left":0.0,"top":0.28092578,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":5,"bounds":{"left":0.013297873,"top":0.29209897,"width":0.4644282,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Pipelines - jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.31364724,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Pipelines - jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.32482043,"width":0.039228722,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.3463687,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.3575419,"width":0.15824468,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Tab","depth":4,"bounds":{"left":0.0028257978,"top":0.38068634,"width":0.07413564,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Customize sidebar","depth":6,"bounds":{"left":0.0028257978,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Close Google Gemini (⌃X)","depth":6,"bounds":{"left":0.013796543,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Tabs from other devices","depth":6,"bounds":{"left":0.024933511,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open history (⇧⌘H)","depth":6,"bounds":{"left":0.036070477,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open bookmarks (⌘B)","depth":6,"bounds":{"left":0.04720745,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"AI Chat settings","depth":7,"bounds":{"left":0.35854387,"top":0.055067837,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Close","depth":7,"bounds":{"left":0.37051198,"top":0.055067837,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Main menu","depth":13,"bounds":{"left":0.08228058,"top":0.09736632,"width":0.015957447,"height":0.03830806},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"New chat","depth":12,"bounds":{"left":0.09823803,"top":0.10215483,"width":0.028091755,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Gemini","depth":15,"bounds":{"left":0.1008976,"top":0.10654429,"width":0.02144282,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Chat","depth":13,"bounds":{"left":0.35854387,"top":0.10215483,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Open menu for conversation actions.","depth":13,"bounds":{"left":0.37051198,"top":0.10215483,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Conversation with Gemini","depth":15,"bounds":{"left":0.079288565,"top":0.14445332,"width":0.0003324468,"height":0.0007980846},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Conversation with Gemini","depth":16,"bounds":{"left":0.079288565,"top":0.14684756,"width":0.1200133,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"You said I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.","depth":23,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Expand","depth":21,"on_screen":false,"role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a concise summary of the changes made in the code selection:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Refactoring and Dependencies","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Refactoring and Dependencies","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Replaced Interface with Implementation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Changed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusInterface","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusManager","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"in the imports and constructor dependency injection.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Redis Support:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Imported","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Illuminate\\Support\\Facades\\Redis","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to handle a newly introduced memory-logging lock mechanism.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Traits:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"WorkerIdTrait","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", along with the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"worker","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"field tracking in the final logs.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Loop and Worker Lifecycle Adjustments","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Loop and Worker Lifecycle Adjustments","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Worker Auto-Restart:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Deleted the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property, the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"constant, and the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"shouldIterateAgain()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method. The script no longer tracks execution iterations or triggers a self-managed worker restart.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Simplified Chunk Method Return:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Modified","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to always return","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Priority Handling for Throttled Entities","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Priority Handling for Throttled Entities","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Stale Record Elevation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Updated the logic in","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"rescheduleThrottledEntities()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to check if a throttled entity group is classified as \"stale\". If it is, the entities are rescheduled using","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"QueuePriorityEnum::NORMAL","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"; otherwise, they keep their respective queue priority.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Metrics, Logging, and Environment Handling","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Metrics, Logging, and Environment Handling","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Environment Tracking:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Introduced a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$environment","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property (defaulting to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"'local'","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") and a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"setEnvironment()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method to dynamically assign it.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Memory Monitoring via Datadog Gauge:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method that submits peak memory allocation to Datadog as a gauge metric (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"jiminny.reindex-memory-usage","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis Lock for Metrics Throttling:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Implemented a Redis-backed lock (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"%s-process-manager-lock","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") with a 60-second expiration inside","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to prevent over-reporting metrics.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Log Payload Update:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added total processed items (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$updatedDocuments + $deletedDocuments","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") to the final information log.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":23,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"You said are there any issues or potential bottlenecks","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"are there any issues or potential bottlenecks","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Edit","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a breakdown of the potential issues to look out for:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"1. Memory Leak Risk (OOM Crashes)","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"1. Memory Leak Risk (OOM Crashes)","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Restart Logic:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The most significant change is the removal of the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Dependency Leaks:","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.050033245,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If any underlying service (like the","depth":26,"bounds":{"left":0.17453457,"top":0.0,"width":0.08277926,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ElasticSearchClient","depth":27,"bounds":{"left":0.25930852,"top":0.0,"width":0.056848403,"height":0.015163607},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", ORM, or Datadog helper) retains references to processed entities,","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.21775267,"height":0.03631285},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"bounds":{"left":0.26811835,"top":0.0,"width":0.056848403,"height":0.015163607},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.23138298,"height":0.05546688},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Hardcoded Limits:","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.047041222,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The code still includes","depth":26,"bounds":{"left":0.17154256,"top":0.0,"width":0.05651596,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ini_set('memory_limit', '512M');","depth":27,"bounds":{"left":0.23005319,"top":0.0,"width":0.095744684,"height":0.015163607},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":". Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.23055187,"height":0.05546688},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"2. Race Condition in Redis Lock","depth":23,"bounds":{"left":0.111369684,"top":0.07940942,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"2. Race Condition in Redis Lock","depth":24,"bounds":{"left":0.111369684,"top":0.079010375,"width":0.09125665,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Non-Atomic Operations:","depth":26,"bounds":{"left":0.12450133,"top":0.10614525,"width":0.062333778,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The","depth":26,"bounds":{"left":0.18683511,"top":0.10614525,"width":0.011635638,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage","depth":27,"bounds":{"left":0.20046543,"top":0.10774142,"width":0.041888297,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method uses three distinct steps to handle the lock:","depth":26,"bounds":{"left":0.12450133,"top":0.10614525,"width":0.22722739,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"get()","depth":27,"bounds":{"left":0.14860372,"top":0.12689546,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":",","depth":26,"bounds":{"left":0.16555852,"top":0.12529927,"width":0.002493351,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"bounds":{"left":0.17004654,"top":0.12689546,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", and","depth":26,"bounds":{"left":0.18700133,"top":0.12529927,"width":0.013131649,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"expire()","depth":27,"bounds":{"left":0.20212767,"top":0.12689546,"width":0.023936171,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"bounds":{"left":0.22805852,"top":0.12529927,"width":0.0013297872,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"bounds":{"left":0.12450133,"top":0.15403032,"width":0.040059842,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If multiple workers process the same","depth":26,"bounds":{"left":0.16456117,"top":0.15403032,"width":0.09275266,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$this->updateTarget","depth":27,"bounds":{"left":0.25930852,"top":0.15562649,"width":0.056848403,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"concurrently, they can both pass the","depth":26,"bounds":{"left":0.12450133,"top":0.15403032,"width":0.22855718,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"if (! is_null(Redis::get(...)))","depth":27,"bounds":{"left":0.18351063,"top":0.17478053,"width":0.09275266,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"check at the exact same millisecond. Both will then execute","depth":26,"bounds":{"left":0.12450133,"top":0.17318435,"width":0.21575798,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"bounds":{"left":0.2122673,"top":0.19393456,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and push duplicate metrics to Datadog.","depth":26,"bounds":{"left":0.22922207,"top":0.19233839,"width":0.09823803,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Fix:","depth":26,"bounds":{"left":0.12450133,"top":0.22106944,"width":0.019281914,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"This should be replaced with a single atomic operation using Redis's","depth":26,"bounds":{"left":0.14378324,"top":0.22106944,"width":0.16821809,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"NX","depth":27,"bounds":{"left":0.31399602,"top":0.22266561,"width":0.005984043,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Not eXists) and","depth":26,"bounds":{"left":0.12450133,"top":0.22106944,"width":0.22855718,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"EX","depth":27,"bounds":{"left":0.13696809,"top":0.24181964,"width":0.005984043,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Expire) flags, or Laravel’s built-in atomic cache locks:","depth":26,"bounds":{"left":0.14494681,"top":0.24022347,"width":0.13264628,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis::set($gaugeLockName, true, 'EX', 60, 'NX')","depth":27,"bounds":{"left":0.12649602,"top":0.26097366,"width":0.14361702,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"bounds":{"left":0.27210772,"top":0.25937748,"width":0.0013297872,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"3. Infinite Loops and Caller Contract","depth":23,"bounds":{"left":0.111369684,"top":0.30925778,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"3. Infinite Loops and Caller Contract","depth":24,"bounds":{"left":0.111369684,"top":0.30885875,"width":0.10571808,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Return Value Change:","depth":26,"bounds":{"left":0.12450133,"top":0.33599362,"width":0.055352394,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"bounds":{"left":0.18301196,"top":0.33758977,"width":0.06881649,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"previously returned whether it should iterate again based on staleness or chunk size. It now always returns","depth":26,"bounds":{"left":0.12450133,"top":0.33599362,"width":0.2237367,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"bounds":{"left":0.29438165,"top":0.3567438,"width":0.011968086,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(unless the selection is totally empty).","depth":26,"bounds":{"left":0.12450133,"top":0.35514766,"width":0.2130984,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"bounds":{"left":0.12450133,"top":0.40303272,"width":0.040059842,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the code calling this method relies on a","depth":26,"bounds":{"left":0.16456117,"top":0.40303272,"width":0.10305851,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"false","depth":27,"bounds":{"left":0.26961437,"top":0.4046289,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"return to pause, sleep, or gracefully terminate, changing this to an unconditional","depth":26,"bounds":{"left":0.12450133,"top":0.40303272,"width":0.22589761,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"bounds":{"left":0.26163563,"top":0.4237829,"width":0.011968086,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"could result in an aggressive \"busy loop\" that spikes CPU usage.","depth":26,"bounds":{"left":0.12450133,"top":0.42218676,"width":0.22290559,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"4. Performance Overhead","depth":23,"bounds":{"left":0.111369684,"top":0.49122107,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"4. Performance Overhead","depth":24,"bounds":{"left":0.111369684,"top":0.49082202,"width":0.076130316,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Expensive Garbage Collection:","depth":26,"bounds":{"left":0.12450133,"top":0.5179569,"width":0.078457445,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Calling","depth":26,"bounds":{"left":0.20295878,"top":0.5179569,"width":0.019281914,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles();","depth":27,"bounds":{"left":0.22423537,"top":0.51955307,"width":0.059840426,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"on","depth":26,"bounds":{"left":0.28607047,"top":0.5179569,"width":0.00880984,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"every single chunk","depth":26,"bounds":{"left":0.29488033,"top":0.5179569,"width":0.045545213,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.","depth":26,"bounds":{"left":0.12450133,"top":0.5179569,"width":0.23420878,"height":0.05546688},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Synchronous Metric Dispatch:","depth":26,"bounds":{"left":0.12450133,"top":0.584996,"width":0.07712766,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the","depth":26,"bounds":{"left":0.20162898,"top":0.584996,"width":0.015292553,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::increment","depth":27,"bounds":{"left":0.21891622,"top":0.5865922,"width":0.053856384,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"or","depth":26,"bounds":{"left":0.27476728,"top":0.584996,"width":0.0078125,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::gauge","depth":27,"bounds":{"left":0.28457448,"top":0.5865922,"width":0.041888297,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.","depth":26,"bounds":{"left":0.12450133,"top":0.584996,"width":0.23420878,"height":0.05546688},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"5. Lack of Explicit Error Handling","depth":23,"bounds":{"left":0.111369684,"top":0.67318434,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5. Lack of Explicit Error Handling","depth":24,"bounds":{"left":0.111369684,"top":0.67278534,"width":0.094913565,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"No","depth":26,"bounds":{"left":0.12450133,"top":0.6999202,"width":0.008477394,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"try/catch","depth":27,"bounds":{"left":0.1349734,"top":0.7015164,"width":0.026928192,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Blocks:","depth":26,"bounds":{"left":0.16389628,"top":0.6999202,"width":0.019614361,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The script executes complex network operations (","depth":26,"bounds":{"left":0.18351063,"top":0.6999202,"width":0.12367021,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"loadDocuments","depth":27,"bounds":{"left":0.30917552,"top":0.7015164,"width":0.038896278,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.","depth":26,"bounds":{"left":0.12450133,"top":0.6999202,"width":0.22888963,"height":0.09377494},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":22,"bounds":{"left":0.1087101,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":22,"bounds":{"left":0.124667555,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Redo","depth":22,"bounds":{"left":0.140625,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":21,"bounds":{"left":0.15658244,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":22,"bounds":{"left":0.17253989,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":21,"bounds":{"left":0.18849733,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXTextArea","text":"Ask Gemini","depth":21,"bounds":{"left":0.14594415,"top":0.86632085,"width":0.17021276,"height":0.01915403},"on_screen":true,"value":"Ask Gemini","help_text":"","role_description":"text entry area","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Ask Gemini","depth":22,"bounds":{"left":0.14594415,"top":0.8667199,"width":0.030418882,"height":0.018355945},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Upload & tools","depth":20,"bounds":{"left":0.12732713,"top":0.8567438,"width":0.015957447,"height":0.03830806},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Microphone","depth":20,"bounds":{"left":0.32413563,"top":0.86153233,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Send message","depth":20,"bounds":{"left":0.33211437,"top":0.8631285,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":false,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.","depth":17,"bounds":{"left":0.102726065,"top":0.9173983,"width":0.21775267,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Your privacy & Gemini Opens in a new window","depth":17,"bounds":{"left":0.32047874,"top":0.9173983,"width":0.043882977,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Your privacy & Gemini","depth":18,"bounds":{"left":0.32047874,"top":0.9173983,"width":0.043882977,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Opens in a new window","depth":19,"bounds":{"left":0.079288565,"top":0.91660017,"width":0.04720745,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Summarize page","depth":7,"bounds":{"left":0.08494016,"top":0.95730245,"width":0.053523935,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Summarize page","depth":9,"bounds":{"left":0.09059176,"top":0.96249,"width":0.042220745,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Skip to content","depth":6,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Skip to content","depth":7,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Open menu","depth":10,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"Homepage (g then d)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"jiminny","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"jiminny","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"app","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false}]...
|
-5942611671819710772
|
-1052937846671460580
|
visual_change
|
accessibility
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo
Share & export
Copy
Show more options
Ask Gemini
Ask Gemini
Upload & tools
Microphone
Send message
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Summarize page
Summarize page
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app...
|
65231
|
NULL
|
NULL
|
NULL
|
|
65233
|
2287
|
0
|
2026-05-20T15:30:16.844616+00:00
|
/Users/lukas/.screenpipe/data/data/2026-05-20/1779 /Users/lukas/.screenpipe/data/data/2026-05-20/1779291016844_m1.jpg...
|
Firefox
|
JY-20915 add alias for EU by LakyLak · Pull Reques JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app — Work...
|
1
|
github.com/jiminny/app/pull/12105
|
monitor_1
|
NULL
|
NULL
|
NULL
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo
Share & export
Copy
Show more options
Ask Gemini
Ask Gemini
Upload & tools
Microphone
Send message
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Summarize page
Summarize page
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
All issues(g then i)
All pull requests
All repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Pull requests (32)
Pull requests
(
32
)
Agents
Agents
Actions
Actions
Wiki
Wiki
Security and quality
Security and quality
Insights
Insights
Settings
Settings
JY-20915 add alias for EU #12105 Edit title
JY-20915 add alias for EU
#
12105
Edit title
Awaiting approval
Awaiting approval
Code
Code
Open
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Lines changed: 365 additions & 1 deletion
Conversation (1)
Conversation
(
1
)
Commits (5)
Commits
(
5
)
Checks (3)
Checks
(
3
)
Files changed (4)
Files changed
(
4
)
Open
JY-20915 add alias for EU #12105 LakyLak wants to merge 5 commits into master from JY-20915-add-domain-specific-email-text-relay Copy head branch name to clipboard
JY-20915 add alias for EU
JY-20915 add alias for EU
#
12105
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Conversation
Conversation
@LakyLak...
|
[{"role":"AXRadioButton","text [{"role":"AXRadioButton","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Close tab","depth":5,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXRadioButton","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Pipelines - jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Pipelines - jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Tab","depth":4,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Customize sidebar","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Close Google Gemini (⌃X)","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Tabs from other devices","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open history (⇧⌘H)","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open bookmarks (⌘B)","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"AI Chat settings","depth":7,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Close","depth":7,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Main menu","depth":13,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"New chat","depth":12,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Gemini","depth":15,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Chat","depth":13,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Open menu for conversation actions.","depth":13,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Conversation with Gemini","depth":15,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Conversation with Gemini","depth":16,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"You said I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.","depth":23,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Expand","depth":21,"on_screen":false,"role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a concise summary of the changes made in the code selection:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Refactoring and Dependencies","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Refactoring and Dependencies","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Replaced Interface with Implementation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Changed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusInterface","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusManager","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"in the imports and constructor dependency injection.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Redis Support:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Imported","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Illuminate\\Support\\Facades\\Redis","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to handle a newly introduced memory-logging lock mechanism.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Traits:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"WorkerIdTrait","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", along with the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"worker","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"field tracking in the final logs.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Loop and Worker Lifecycle Adjustments","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Loop and Worker Lifecycle Adjustments","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Worker Auto-Restart:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Deleted the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property, the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"constant, and the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"shouldIterateAgain()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method. The script no longer tracks execution iterations or triggers a self-managed worker restart.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Simplified Chunk Method Return:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Modified","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to always return","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Priority Handling for Throttled Entities","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Priority Handling for Throttled Entities","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Stale Record Elevation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Updated the logic in","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"rescheduleThrottledEntities()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to check if a throttled entity group is classified as \"stale\". If it is, the entities are rescheduled using","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"QueuePriorityEnum::NORMAL","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"; otherwise, they keep their respective queue priority.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Metrics, Logging, and Environment Handling","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Metrics, Logging, and Environment Handling","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Environment Tracking:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Introduced a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$environment","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property (defaulting to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"'local'","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") and a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"setEnvironment()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method to dynamically assign it.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Memory Monitoring via Datadog Gauge:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method that submits peak memory allocation to Datadog as a gauge metric (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"jiminny.reindex-memory-usage","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis Lock for Metrics Throttling:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Implemented a Redis-backed lock (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"%s-process-manager-lock","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") with a 60-second expiration inside","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to prevent over-reporting metrics.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Log Payload Update:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added total processed items (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$updatedDocuments + $deletedDocuments","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") to the final information log.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":23,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"You said are there any issues or potential bottlenecks","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"are there any issues or potential bottlenecks","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Edit","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a breakdown of the potential issues to look out for:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"1. Memory Leak Risk (OOM Crashes)","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"1. Memory Leak Risk (OOM Crashes)","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Restart Logic:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The most significant change is the removal of the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Dependency Leaks:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If any underlying service (like the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ElasticSearchClient","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", ORM, or Datadog helper) retains references to processed entities,","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Hardcoded Limits:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The code still includes","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ini_set('memory_limit', '512M');","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":". Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"2. Race Condition in Redis Lock","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"2. Race Condition in Redis Lock","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Non-Atomic Operations:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method uses three distinct steps to handle the lock:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"get()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":",","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", and","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"expire()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If multiple workers process the same","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$this->updateTarget","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"concurrently, they can both pass the","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"if (! is_null(Redis::get(...)))","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"check at the exact same millisecond. Both will then execute","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and push duplicate metrics to Datadog.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Fix:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"This should be replaced with a single atomic operation using Redis's","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"NX","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Not eXists) and","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"EX","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Expire) flags, or Laravel’s built-in atomic cache locks:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis::set($gaugeLockName, true, 'EX', 60, 'NX')","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"3. Infinite Loops and Caller Contract","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"3. Infinite Loops and Caller Contract","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Return Value Change:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"previously returned whether it should iterate again based on staleness or chunk size. It now always returns","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(unless the selection is totally empty).","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the code calling this method relies on a","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"false","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"return to pause, sleep, or gracefully terminate, changing this to an unconditional","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"could result in an aggressive \"busy loop\" that spikes CPU usage.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"4. Performance Overhead","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"4. Performance Overhead","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Expensive Garbage Collection:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Calling","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles();","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"on","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"every single chunk","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Synchronous Metric Dispatch:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::increment","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"or","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::gauge","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"5. Lack of Explicit Error Handling","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5. Lack of Explicit Error Handling","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"No","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"try/catch","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Blocks:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The script executes complex network operations (","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"loadDocuments","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":22,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":22,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Redo","depth":22,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":21,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":22,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":21,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXTextArea","text":"Ask Gemini","depth":21,"on_screen":true,"value":"Ask Gemini","help_text":"","role_description":"text entry area","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Ask Gemini","depth":22,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Upload & tools","depth":20,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Microphone","depth":20,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Send message","depth":20,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":false,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.","depth":17,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Your privacy & Gemini Opens in a new window","depth":17,"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Your privacy & Gemini","depth":18,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Opens in a new window","depth":19,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Summarize page","depth":7,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Summarize page","depth":9,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Skip to content","depth":6,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Skip to content","depth":7,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Open menu","depth":10,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"Homepage (g then d)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"jiminny","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"jiminny","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"app","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"app","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Search or jump to…","depth":9,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Type","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"/","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to search","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Chat with Copilot","depth":10,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXMenuButton","text":"Open Copilot…","depth":9,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXMenuButton","text":"Create new...","depth":9,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"All issues(g then i)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"All pull requests","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"All repositories","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"You have unread notifications(g then n)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Open user navigation menu","depth":9,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Repository navigation","depth":9,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Repository navigation","depth":10,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Code","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Code","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Pull requests (32)","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Pull requests","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"32","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Agents","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Agents","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Actions","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Actions","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Wiki","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Wiki","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Security and quality","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Security and quality","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Insights","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Insights","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Settings","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Settings","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"JY-20915 add alias for EU #12105 Edit title","depth":13,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"JY-20915 add alias for EU","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"#","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"12105","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Edit title","depth":14,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Awaiting approval","depth":13,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Awaiting approval","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXMenuButton","text":"Code","depth":13,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXStaticText","text":"Code","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Open","depth":13,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"LakyLak","depth":15,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"wants to merge 5 commits into","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"master","depth":15,"on_screen":false,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"master","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"from","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915-add-domain-specific-email-text-relay","depth":16,"on_screen":false,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915-add-domain-specific-email-text-relay","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy head branch name to clipboard","depth":16,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Lines changed: 365 additions & 1 deletion","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Conversation (1)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"Conversation","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"1","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Commits (5)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Commits","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Checks (3)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Checks","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"3","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Files changed (4)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Files changed","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"4","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Open","depth":14,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"JY-20915 add alias for EU #12105 LakyLak wants to merge 5 commits into master from JY-20915-add-domain-specific-email-text-relay Copy head branch name to clipboard","depth":14,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915 add alias for EU","depth":16,"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915 add alias for EU","depth":17,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"#","depth":16,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"12105","depth":16,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"LakyLak","depth":18,"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":19,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"wants to merge 5 commits into","depth":18,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"master","depth":18,"on_screen":true,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"master","depth":19,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"from","depth":19,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915-add-domain-specific-email-text-relay","depth":19,"on_screen":true,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915-add-domain-specific-email-text-relay","depth":20,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy head branch name to clipboard","depth":19,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Conversation","depth":12,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Conversation","depth":13,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@LakyLak","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false}]...
|
2188971022416912507
|
-1048292410045221092
|
click
|
accessibility
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo
Share & export
Copy
Show more options
Ask Gemini
Ask Gemini
Upload & tools
Microphone
Send message
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Summarize page
Summarize page
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
All issues(g then i)
All pull requests
All repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Pull requests (32)
Pull requests
(
32
)
Agents
Agents
Actions
Actions
Wiki
Wiki
Security and quality
Security and quality
Insights
Insights
Settings
Settings
JY-20915 add alias for EU #12105 Edit title
JY-20915 add alias for EU
#
12105
Edit title
Awaiting approval
Awaiting approval
Code
Code
Open
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Lines changed: 365 additions & 1 deletion
Conversation (1)
Conversation
(
1
)
Commits (5)
Commits
(
5
)
Checks (3)
Checks
(
3
)
Files changed (4)
Files changed
(
4
)
Open
JY-20915 add alias for EU #12105 LakyLak wants to merge 5 commits into master from JY-20915-add-domain-specific-email-text-relay Copy head branch name to clipboard
JY-20915 add alias for EU
JY-20915 add alias for EU
#
12105
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Conversation
Conversation
@LakyLak...
|
65226
|
NULL
|
NULL
|
NULL
|
|
65234
|
2288
|
0
|
2026-05-20T15:30:28.172074+00:00
|
/Users/lukas/.screenpipe/data/data/2026-05-20/1779 /Users/lukas/.screenpipe/data/data/2026-05-20/1779291028172_m2.jpg...
|
Firefox
|
JY-20915 add alias for EU by LakyLak · Pull Reques JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app — Work...
|
1
|
github.com/jiminny/app/pull/12105
|
monitor_2
|
NULL
|
NULL
|
NULL
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo
Share & export
Copy
Show more options
Ask Gemini
Ask Gemini
Upload & tools
Microphone
Send message
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Summarize page
Summarize page
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
All issues(g then i)
All pull requests
All repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Pull requests (32)
Pull requests
(
32
)
Agents
Agents
Actions
Actions
Wiki
Wiki
Security and quality
Security and quality
Insights
Insights
Settings
Settings
JY-20915 add alias for EU #12105 Edit title
JY-20915 add alias for EU
#
12105
Edit title
Awaiting approval
Awaiting approval
Code...
|
[{"role":"AXRadioButton","text [{"role":"AXRadioButton","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":4,"bounds":{"left":0.0,"top":0.0518755,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.06304868,"width":0.10106383,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":4,"bounds":{"left":0.0,"top":0.08459697,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.09577015,"width":0.10721409,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.11731844,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.12849163,"width":0.15791224,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.15003991,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.16121309,"width":0.1796875,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.18276137,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.19393456,"width":0.12699468,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Close tab","depth":5,"bounds":{"left":0.06732048,"top":0.18994413,"width":0.007978723,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXRadioButton","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":4,"bounds":{"left":0.0,"top":0.21548285,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":5,"bounds":{"left":0.013297873,"top":0.22665602,"width":0.10139628,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":4,"bounds":{"left":0.0,"top":0.2482043,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.25937748,"width":0.18068483,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":4,"bounds":{"left":0.0,"top":0.28092578,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":5,"bounds":{"left":0.013297873,"top":0.29209897,"width":0.4644282,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Pipelines - jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.31364724,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Pipelines - jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.32482043,"width":0.039228722,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.3463687,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.3575419,"width":0.15824468,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Tab","depth":4,"bounds":{"left":0.0028257978,"top":0.38068634,"width":0.07413564,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Customize sidebar","depth":6,"bounds":{"left":0.0028257978,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Close Google Gemini (⌃X)","depth":6,"bounds":{"left":0.013796543,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Tabs from other devices","depth":6,"bounds":{"left":0.024933511,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open history (⇧⌘H)","depth":6,"bounds":{"left":0.036070477,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open bookmarks (⌘B)","depth":6,"bounds":{"left":0.04720745,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"AI Chat settings","depth":7,"bounds":{"left":0.35854387,"top":0.055067837,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Close","depth":7,"bounds":{"left":0.37051198,"top":0.055067837,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Main menu","depth":13,"bounds":{"left":0.08228058,"top":0.09736632,"width":0.015957447,"height":0.03830806},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"New chat","depth":12,"bounds":{"left":0.09823803,"top":0.10215483,"width":0.028091755,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Gemini","depth":15,"bounds":{"left":0.1008976,"top":0.10654429,"width":0.02144282,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Chat","depth":13,"bounds":{"left":0.35854387,"top":0.10215483,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Open menu for conversation actions.","depth":13,"bounds":{"left":0.37051198,"top":0.10215483,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Conversation with Gemini","depth":15,"bounds":{"left":0.079288565,"top":0.14445332,"width":0.0003324468,"height":0.0007980846},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Conversation with Gemini","depth":16,"bounds":{"left":0.079288565,"top":0.14684756,"width":0.1200133,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"You said I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.","depth":23,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Expand","depth":21,"on_screen":false,"role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a concise summary of the changes made in the code selection:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Refactoring and Dependencies","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Refactoring and Dependencies","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Replaced Interface with Implementation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Changed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusInterface","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusManager","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"in the imports and constructor dependency injection.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Redis Support:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Imported","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Illuminate\\Support\\Facades\\Redis","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to handle a newly introduced memory-logging lock mechanism.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Traits:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"WorkerIdTrait","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", along with the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"worker","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"field tracking in the final logs.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Loop and Worker Lifecycle Adjustments","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Loop and Worker Lifecycle Adjustments","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Worker Auto-Restart:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Deleted the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property, the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"constant, and the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"shouldIterateAgain()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method. The script no longer tracks execution iterations or triggers a self-managed worker restart.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Simplified Chunk Method Return:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Modified","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to always return","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Priority Handling for Throttled Entities","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Priority Handling for Throttled Entities","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Stale Record Elevation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Updated the logic in","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"rescheduleThrottledEntities()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to check if a throttled entity group is classified as \"stale\". If it is, the entities are rescheduled using","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"QueuePriorityEnum::NORMAL","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"; otherwise, they keep their respective queue priority.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Metrics, Logging, and Environment Handling","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Metrics, Logging, and Environment Handling","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Environment Tracking:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Introduced a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$environment","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property (defaulting to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"'local'","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") and a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"setEnvironment()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method to dynamically assign it.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Memory Monitoring via Datadog Gauge:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method that submits peak memory allocation to Datadog as a gauge metric (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"jiminny.reindex-memory-usage","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis Lock for Metrics Throttling:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Implemented a Redis-backed lock (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"%s-process-manager-lock","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") with a 60-second expiration inside","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to prevent over-reporting metrics.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Log Payload Update:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added total processed items (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$updatedDocuments + $deletedDocuments","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") to the final information log.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":23,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"You said are there any issues or potential bottlenecks","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"are there any issues or potential bottlenecks","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Edit","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a breakdown of the potential issues to look out for:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"1. Memory Leak Risk (OOM Crashes)","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"1. Memory Leak Risk (OOM Crashes)","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Restart Logic:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The most significant change is the removal of the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Dependency Leaks:","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.050033245,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If any underlying service (like the","depth":26,"bounds":{"left":0.17453457,"top":0.0,"width":0.08277926,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ElasticSearchClient","depth":27,"bounds":{"left":0.25930852,"top":0.0,"width":0.056848403,"height":0.015163607},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", ORM, or Datadog helper) retains references to processed entities,","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.21775267,"height":0.03631285},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"bounds":{"left":0.26811835,"top":0.0,"width":0.056848403,"height":0.015163607},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.23138298,"height":0.05546688},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Hardcoded Limits:","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.047041222,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The code still includes","depth":26,"bounds":{"left":0.17154256,"top":0.0,"width":0.05651596,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ini_set('memory_limit', '512M');","depth":27,"bounds":{"left":0.23005319,"top":0.0,"width":0.095744684,"height":0.015163607},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":". Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.23055187,"height":0.05546688},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"2. Race Condition in Redis Lock","depth":23,"bounds":{"left":0.111369684,"top":0.07940942,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"2. Race Condition in Redis Lock","depth":24,"bounds":{"left":0.111369684,"top":0.079010375,"width":0.09125665,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Non-Atomic Operations:","depth":26,"bounds":{"left":0.12450133,"top":0.10614525,"width":0.062333778,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The","depth":26,"bounds":{"left":0.18683511,"top":0.10614525,"width":0.011635638,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage","depth":27,"bounds":{"left":0.20046543,"top":0.10774142,"width":0.041888297,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method uses three distinct steps to handle the lock:","depth":26,"bounds":{"left":0.12450133,"top":0.10614525,"width":0.22722739,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"get()","depth":27,"bounds":{"left":0.14860372,"top":0.12689546,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":",","depth":26,"bounds":{"left":0.16555852,"top":0.12529927,"width":0.002493351,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"bounds":{"left":0.17004654,"top":0.12689546,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", and","depth":26,"bounds":{"left":0.18700133,"top":0.12529927,"width":0.013131649,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"expire()","depth":27,"bounds":{"left":0.20212767,"top":0.12689546,"width":0.023936171,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"bounds":{"left":0.22805852,"top":0.12529927,"width":0.0013297872,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"bounds":{"left":0.12450133,"top":0.15403032,"width":0.040059842,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If multiple workers process the same","depth":26,"bounds":{"left":0.16456117,"top":0.15403032,"width":0.09275266,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$this->updateTarget","depth":27,"bounds":{"left":0.25930852,"top":0.15562649,"width":0.056848403,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"concurrently, they can both pass the","depth":26,"bounds":{"left":0.12450133,"top":0.15403032,"width":0.22855718,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"if (! is_null(Redis::get(...)))","depth":27,"bounds":{"left":0.18351063,"top":0.17478053,"width":0.09275266,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"check at the exact same millisecond. Both will then execute","depth":26,"bounds":{"left":0.12450133,"top":0.17318435,"width":0.21575798,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"bounds":{"left":0.2122673,"top":0.19393456,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and push duplicate metrics to Datadog.","depth":26,"bounds":{"left":0.22922207,"top":0.19233839,"width":0.09823803,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Fix:","depth":26,"bounds":{"left":0.12450133,"top":0.22106944,"width":0.019281914,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"This should be replaced with a single atomic operation using Redis's","depth":26,"bounds":{"left":0.14378324,"top":0.22106944,"width":0.16821809,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"NX","depth":27,"bounds":{"left":0.31399602,"top":0.22266561,"width":0.005984043,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Not eXists) and","depth":26,"bounds":{"left":0.12450133,"top":0.22106944,"width":0.22855718,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"EX","depth":27,"bounds":{"left":0.13696809,"top":0.24181964,"width":0.005984043,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Expire) flags, or Laravel’s built-in atomic cache locks:","depth":26,"bounds":{"left":0.14494681,"top":0.24022347,"width":0.13264628,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis::set($gaugeLockName, true, 'EX', 60, 'NX')","depth":27,"bounds":{"left":0.12649602,"top":0.26097366,"width":0.14361702,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"bounds":{"left":0.27210772,"top":0.25937748,"width":0.0013297872,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"3. Infinite Loops and Caller Contract","depth":23,"bounds":{"left":0.111369684,"top":0.30925778,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"3. Infinite Loops and Caller Contract","depth":24,"bounds":{"left":0.111369684,"top":0.30885875,"width":0.10571808,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Return Value Change:","depth":26,"bounds":{"left":0.12450133,"top":0.33599362,"width":0.055352394,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"bounds":{"left":0.18301196,"top":0.33758977,"width":0.06881649,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"previously returned whether it should iterate again based on staleness or chunk size. It now always returns","depth":26,"bounds":{"left":0.12450133,"top":0.33599362,"width":0.2237367,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"bounds":{"left":0.29438165,"top":0.3567438,"width":0.011968086,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(unless the selection is totally empty).","depth":26,"bounds":{"left":0.12450133,"top":0.35514766,"width":0.2130984,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"bounds":{"left":0.12450133,"top":0.40303272,"width":0.040059842,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the code calling this method relies on a","depth":26,"bounds":{"left":0.16456117,"top":0.40303272,"width":0.10305851,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"false","depth":27,"bounds":{"left":0.26961437,"top":0.4046289,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"return to pause, sleep, or gracefully terminate, changing this to an unconditional","depth":26,"bounds":{"left":0.12450133,"top":0.40303272,"width":0.22589761,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"bounds":{"left":0.26163563,"top":0.4237829,"width":0.011968086,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"could result in an aggressive \"busy loop\" that spikes CPU usage.","depth":26,"bounds":{"left":0.12450133,"top":0.42218676,"width":0.22290559,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"4. Performance Overhead","depth":23,"bounds":{"left":0.111369684,"top":0.49122107,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"4. Performance Overhead","depth":24,"bounds":{"left":0.111369684,"top":0.49082202,"width":0.076130316,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Expensive Garbage Collection:","depth":26,"bounds":{"left":0.12450133,"top":0.5179569,"width":0.078457445,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Calling","depth":26,"bounds":{"left":0.20295878,"top":0.5179569,"width":0.019281914,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles();","depth":27,"bounds":{"left":0.22423537,"top":0.51955307,"width":0.059840426,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"on","depth":26,"bounds":{"left":0.28607047,"top":0.5179569,"width":0.00880984,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"every single chunk","depth":26,"bounds":{"left":0.29488033,"top":0.5179569,"width":0.045545213,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.","depth":26,"bounds":{"left":0.12450133,"top":0.5179569,"width":0.23420878,"height":0.05546688},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Synchronous Metric Dispatch:","depth":26,"bounds":{"left":0.12450133,"top":0.584996,"width":0.07712766,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the","depth":26,"bounds":{"left":0.20162898,"top":0.584996,"width":0.015292553,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::increment","depth":27,"bounds":{"left":0.21891622,"top":0.5865922,"width":0.053856384,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"or","depth":26,"bounds":{"left":0.27476728,"top":0.584996,"width":0.0078125,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::gauge","depth":27,"bounds":{"left":0.28457448,"top":0.5865922,"width":0.041888297,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.","depth":26,"bounds":{"left":0.12450133,"top":0.584996,"width":0.23420878,"height":0.05546688},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"5. Lack of Explicit Error Handling","depth":23,"bounds":{"left":0.111369684,"top":0.67318434,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5. Lack of Explicit Error Handling","depth":24,"bounds":{"left":0.111369684,"top":0.67278534,"width":0.094913565,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"No","depth":26,"bounds":{"left":0.12450133,"top":0.6999202,"width":0.008477394,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"try/catch","depth":27,"bounds":{"left":0.1349734,"top":0.7015164,"width":0.026928192,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Blocks:","depth":26,"bounds":{"left":0.16389628,"top":0.6999202,"width":0.019614361,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The script executes complex network operations (","depth":26,"bounds":{"left":0.18351063,"top":0.6999202,"width":0.12367021,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"loadDocuments","depth":27,"bounds":{"left":0.30917552,"top":0.7015164,"width":0.038896278,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.","depth":26,"bounds":{"left":0.12450133,"top":0.6999202,"width":0.22888963,"height":0.09377494},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":22,"bounds":{"left":0.1087101,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":22,"bounds":{"left":0.124667555,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Redo","depth":22,"bounds":{"left":0.140625,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":21,"bounds":{"left":0.15658244,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":22,"bounds":{"left":0.17253989,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":21,"bounds":{"left":0.18849733,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXTextArea","text":"Ask Gemini","depth":21,"bounds":{"left":0.14594415,"top":0.86632085,"width":0.17021276,"height":0.01915403},"on_screen":true,"value":"Ask Gemini","help_text":"","role_description":"text entry area","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Ask Gemini","depth":22,"bounds":{"left":0.14594415,"top":0.8667199,"width":0.030418882,"height":0.018355945},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Upload & tools","depth":20,"bounds":{"left":0.12732713,"top":0.8567438,"width":0.015957447,"height":0.03830806},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Microphone","depth":20,"bounds":{"left":0.32413563,"top":0.86153233,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Send message","depth":20,"bounds":{"left":0.33211437,"top":0.8631285,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":false,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.","depth":17,"bounds":{"left":0.102726065,"top":0.9173983,"width":0.21775267,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Your privacy & Gemini Opens in a new window","depth":17,"bounds":{"left":0.32047874,"top":0.9173983,"width":0.043882977,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Your privacy & Gemini","depth":18,"bounds":{"left":0.32047874,"top":0.9173983,"width":0.043882977,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Opens in a new window","depth":19,"bounds":{"left":0.079288565,"top":0.91660017,"width":0.04720745,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Summarize page","depth":7,"bounds":{"left":0.08494016,"top":0.95730245,"width":0.053523935,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Summarize page","depth":9,"bounds":{"left":0.09059176,"top":0.96249,"width":0.042220745,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Skip to content","depth":6,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Skip to content","depth":7,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Open menu","depth":10,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"Homepage (g then d)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"jiminny","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"jiminny","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"app","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"app","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Search or jump to…","depth":9,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Type","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"/","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to search","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Chat with Copilot","depth":10,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXMenuButton","text":"Open Copilot…","depth":9,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXMenuButton","text":"Create new...","depth":9,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"All issues(g then i)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"All pull requests","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"All repositories","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"You have unread notifications(g then n)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Open user navigation menu","depth":9,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Repository navigation","depth":9,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Repository navigation","depth":10,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Code","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Code","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Pull requests (32)","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Pull requests","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"32","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Agents","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Agents","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Actions","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Actions","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Wiki","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Wiki","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Security and quality","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Security and quality","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Insights","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Insights","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Settings","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Settings","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"JY-20915 add alias for EU #12105 Edit title","depth":13,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"JY-20915 add alias for EU","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"#","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"12105","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Edit title","depth":14,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Awaiting approval","depth":13,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Awaiting approval","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXMenuButton","text":"Code","depth":13,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false}]...
|
94103359384234457
|
-1052725640927283428
|
idle
|
accessibility
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo
Share & export
Copy
Show more options
Ask Gemini
Ask Gemini
Upload & tools
Microphone
Send message
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Summarize page
Summarize page
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
All issues(g then i)
All pull requests
All repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Pull requests (32)
Pull requests
(
32
)
Agents
Agents
Actions
Actions
Wiki
Wiki
Security and quality
Security and quality
Insights
Insights
Settings
Settings
JY-20915 add alias for EU #12105 Edit title
JY-20915 add alias for EU
#
12105
Edit title
Awaiting approval
Awaiting approval
Code...
|
NULL
|
NULL
|
NULL
|
NULL
|
|
65235
|
2287
|
1
|
2026-05-20T15:30:47.232514+00:00
|
/Users/lukas/.screenpipe/data/data/2026-05-20/1779 /Users/lukas/.screenpipe/data/data/2026-05-20/1779291047232_m1.jpg...
|
Firefox
|
JY-20915 add alias for EU by LakyLak · Pull Reques JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app — Work...
|
1
|
github.com/jiminny/app/pull/12105
|
monitor_1
|
NULL
|
NULL
|
NULL
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:...
|
[{"role":"AXRadioButton","text [{"role":"AXRadioButton","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Close tab","depth":5,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXRadioButton","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Pipelines - jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Pipelines - jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Tab","depth":4,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Customize sidebar","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Close Google Gemini (⌃X)","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Tabs from other devices","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open history (⇧⌘H)","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open bookmarks (⌘B)","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"AI Chat settings","depth":7,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Close","depth":7,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Main menu","depth":13,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"New chat","depth":12,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Gemini","depth":15,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Chat","depth":13,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Open menu for conversation actions.","depth":13,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Conversation with Gemini","depth":15,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Conversation with Gemini","depth":16,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"You said I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.","depth":23,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Expand","depth":21,"on_screen":false,"role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a concise summary of the changes made in the code selection:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Refactoring and Dependencies","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Refactoring and Dependencies","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Replaced Interface with Implementation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Changed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusInterface","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusManager","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"in the imports and constructor dependency injection.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Redis Support:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Imported","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Illuminate\\Support\\Facades\\Redis","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to handle a newly introduced memory-logging lock mechanism.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Traits:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"WorkerIdTrait","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", along with the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"worker","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"field tracking in the final logs.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Loop and Worker Lifecycle Adjustments","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Loop and Worker Lifecycle Adjustments","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Worker Auto-Restart:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Deleted the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property, the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"constant, and the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"shouldIterateAgain()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method. The script no longer tracks execution iterations or triggers a self-managed worker restart.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Simplified Chunk Method Return:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Modified","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to always return","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Priority Handling for Throttled Entities","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Priority Handling for Throttled Entities","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Stale Record Elevation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Updated the logic in","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"rescheduleThrottledEntities()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to check if a throttled entity group is classified as \"stale\". If it is, the entities are rescheduled using","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"QueuePriorityEnum::NORMAL","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"; otherwise, they keep their respective queue priority.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Metrics, Logging, and Environment Handling","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Metrics, Logging, and Environment Handling","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Environment Tracking:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Introduced a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$environment","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property (defaulting to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"'local'","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") and a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"setEnvironment()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method to dynamically assign it.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Memory Monitoring via Datadog Gauge:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method that submits peak memory allocation to Datadog as a gauge metric (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"jiminny.reindex-memory-usage","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis Lock for Metrics Throttling:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Implemented a Redis-backed lock (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"%s-process-manager-lock","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") with a 60-second expiration inside","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to prevent over-reporting metrics.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Log Payload Update:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added total processed items (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$updatedDocuments + $deletedDocuments","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") to the final information log.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":23,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"You said are there any issues or potential bottlenecks","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"are there any issues or potential bottlenecks","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Edit","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a breakdown of the potential issues to look out for:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"1. Memory Leak Risk (OOM Crashes)","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"1. Memory Leak Risk (OOM Crashes)","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Restart Logic:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The most significant change is the removal of the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Dependency Leaks:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If any underlying service (like the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ElasticSearchClient","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", ORM, or Datadog helper) retains references to processed entities,","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Hardcoded Limits:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The code still includes","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ini_set('memory_limit', '512M');","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":". Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"2. Race Condition in Redis Lock","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"2. Race Condition in Redis Lock","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Non-Atomic Operations:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method uses three distinct steps to handle the lock:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"get()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":",","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", and","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"expire()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If multiple workers process the same","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$this->updateTarget","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"concurrently, they can both pass the","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"if (! is_null(Redis::get(...)))","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"check at the exact same millisecond. Both will then execute","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and push duplicate metrics to Datadog.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Fix:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"This should be replaced with a single atomic operation using Redis's","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"NX","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Not eXists) and","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"EX","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Expire) flags, or Laravel’s built-in atomic cache locks:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis::set($gaugeLockName, true, 'EX', 60, 'NX')","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"3. Infinite Loops and Caller Contract","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"3. Infinite Loops and Caller Contract","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Return Value Change:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"previously returned whether it should iterate again based on staleness or chunk size. It now always returns","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(unless the selection is totally empty).","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the code calling this method relies on a","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"false","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"return to pause, sleep, or gracefully terminate, changing this to an unconditional","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"could result in an aggressive \"busy loop\" that spikes CPU usage.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"4. Performance Overhead","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"4. Performance Overhead","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Expensive Garbage Collection:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"}]...
|
5551992334650284311
|
-1079889110047033380
|
idle
|
accessibility
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:...
|
NULL
|
NULL
|
NULL
|
NULL
|
|
65236
|
2288
|
1
|
2026-05-20T15:30:58.648219+00:00
|
/Users/lukas/.screenpipe/data/data/2026-05-20/1779 /Users/lukas/.screenpipe/data/data/2026-05-20/1779291058648_m2.jpg...
|
Firefox
|
JY-20915 add alias for EU by LakyLak · Pull Reques JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app — Work...
|
1
|
github.com/jiminny/app/pull/12105
|
monitor_2
|
NULL
|
NULL
|
NULL
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo
Share & export
Copy
Show more options
Ask Gemini
Ask Gemini
Upload & tools
Microphone
Send message
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Summarize page
Summarize page
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
All issues(g then i)
All pull requests
All repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Pull requests (32)
Pull requests
(
32
)
Agents
Agents
Actions
Actions
Wiki
Wiki
Security and quality
Security and quality
Insights
Insights
Settings
Settings
JY-20915 add alias for EU #12105 Edit title
JY-20915 add alias for EU
#
12105
Edit title
Awaiting approval
Awaiting approval
Code
Code
Open
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Lines changed: 365 additions & 1 deletion
Conversation (1)
Conversation
(
1
)
Commits (5)
Commits
(
5
)
Checks (3)
Checks
(
3
)
Files changed (4)
Files changed
(
4
)
Open
JY-20915 add alias for EU #12105 LakyLak wants to merge 5 commits into master from JY-20915-add-domain-specific-email-text-relay Copy head branch name to clipboard
JY-20915 add alias for EU
JY-20915 add alias for EU
#
12105
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Conversation
Conversation
@LakyLak
Show options
LakyLak commented yesterday
LakyLak
LakyLak
commented
yesterday
yesterday
JIRA: JY-20915
JIRA:
JY-20915
JY-20915
Changes:
Changes:
Support text relay for EU
Prevent errors from EU on US
Add or remove reactions
LakyLak
LakyLak
added
5
commits
yesterday
yesterday
@LakyLak
JY-20915
JY-20915
add alias for EU
add alias for EU
8 / 10 checks OK
e5255d5
e5255d5
@LakyLak
JY-20915
JY-20915
fix env recognization
fix env recognization
11 / 12 checks OK
8953054
8953054
@LakyLak
JY-20915
JY-20915
code smells & tests
code smells & tests
11 / 12 checks OK
73d4bea
73d4bea
@LakyLak
JY-20915
JY-20915
add more tests
add more tests
11 / 12 checks OK
5d2dd3e
5d2dd3e
@LakyLak
JY-20915
JY-20915
add sms received test
add sms received test
12 / 12 checks OK
b8c8eb5
b8c8eb5
@sonarqubecloud
Show options
sonarqubecloud Bot commented 3 hours ago
sonarqubecloud
sonarqubecloud
Bot
commented
3 hours ago
3 hours ago
Quality Gate Passed Quality Gate passed
Quality Gate Passed
Quality Gate passed
Issues
0 New issues
0 New issues
0 Accepted issues
0 Accepted issues
Measures
0 Security Hotspots
0 Security Hotspots
72.2% Coverage on New Code
72.2% Coverage on New Code
0.0% Duplication on New Code
0.0% Duplication on New Code...
|
[{"role":"AXRadioButton","text [{"role":"AXRadioButton","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":4,"bounds":{"left":0.0,"top":0.0518755,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.06304868,"width":0.10106383,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":4,"bounds":{"left":0.0,"top":0.08459697,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.09577015,"width":0.10721409,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.11731844,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.12849163,"width":0.15791224,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.15003991,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.16121309,"width":0.1796875,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.18276137,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.19393456,"width":0.12699468,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Close tab","depth":5,"bounds":{"left":0.06732048,"top":0.18994413,"width":0.007978723,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXRadioButton","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":4,"bounds":{"left":0.0,"top":0.21548285,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":5,"bounds":{"left":0.013297873,"top":0.22665602,"width":0.10139628,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":4,"bounds":{"left":0.0,"top":0.2482043,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.25937748,"width":0.18068483,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":4,"bounds":{"left":0.0,"top":0.28092578,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":5,"bounds":{"left":0.013297873,"top":0.29209897,"width":0.4644282,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Pipelines - jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.31364724,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Pipelines - jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.32482043,"width":0.039228722,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.3463687,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.3575419,"width":0.15824468,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Tab","depth":4,"bounds":{"left":0.0028257978,"top":0.38068634,"width":0.07413564,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Customize sidebar","depth":6,"bounds":{"left":0.0028257978,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Close Google Gemini (⌃X)","depth":6,"bounds":{"left":0.013796543,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Tabs from other devices","depth":6,"bounds":{"left":0.024933511,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open history (⇧⌘H)","depth":6,"bounds":{"left":0.036070477,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open bookmarks (⌘B)","depth":6,"bounds":{"left":0.04720745,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"AI Chat settings","depth":7,"bounds":{"left":0.35854387,"top":0.055067837,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Close","depth":7,"bounds":{"left":0.37051198,"top":0.055067837,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Main menu","depth":13,"bounds":{"left":0.08228058,"top":0.09736632,"width":0.015957447,"height":0.03830806},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"New chat","depth":12,"bounds":{"left":0.09823803,"top":0.10215483,"width":0.028091755,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Gemini","depth":15,"bounds":{"left":0.1008976,"top":0.10654429,"width":0.02144282,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Chat","depth":13,"bounds":{"left":0.35854387,"top":0.10215483,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Open menu for conversation actions.","depth":13,"bounds":{"left":0.37051198,"top":0.10215483,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Conversation with Gemini","depth":15,"bounds":{"left":0.079288565,"top":0.14445332,"width":0.0003324468,"height":0.0007980846},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Conversation with Gemini","depth":16,"bounds":{"left":0.079288565,"top":0.14684756,"width":0.1200133,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"You said I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.","depth":23,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Expand","depth":21,"on_screen":false,"role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a concise summary of the changes made in the code selection:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Refactoring and Dependencies","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Refactoring and Dependencies","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Replaced Interface with Implementation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Changed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusInterface","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusManager","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"in the imports and constructor dependency injection.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Redis Support:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Imported","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Illuminate\\Support\\Facades\\Redis","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to handle a newly introduced memory-logging lock mechanism.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Traits:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"WorkerIdTrait","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", along with the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"worker","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"field tracking in the final logs.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Loop and Worker Lifecycle Adjustments","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Loop and Worker Lifecycle Adjustments","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Worker Auto-Restart:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Deleted the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property, the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"constant, and the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"shouldIterateAgain()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method. The script no longer tracks execution iterations or triggers a self-managed worker restart.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Simplified Chunk Method Return:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Modified","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to always return","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Priority Handling for Throttled Entities","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Priority Handling for Throttled Entities","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Stale Record Elevation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Updated the logic in","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"rescheduleThrottledEntities()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to check if a throttled entity group is classified as \"stale\". If it is, the entities are rescheduled using","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"QueuePriorityEnum::NORMAL","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"; otherwise, they keep their respective queue priority.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Metrics, Logging, and Environment Handling","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Metrics, Logging, and Environment Handling","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Environment Tracking:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Introduced a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$environment","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property (defaulting to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"'local'","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") and a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"setEnvironment()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method to dynamically assign it.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Memory Monitoring via Datadog Gauge:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method that submits peak memory allocation to Datadog as a gauge metric (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"jiminny.reindex-memory-usage","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis Lock for Metrics Throttling:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Implemented a Redis-backed lock (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"%s-process-manager-lock","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") with a 60-second expiration inside","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to prevent over-reporting metrics.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Log Payload Update:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added total processed items (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$updatedDocuments + $deletedDocuments","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") to the final information log.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":23,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"You said are there any issues or potential bottlenecks","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"are there any issues or potential bottlenecks","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Edit","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a breakdown of the potential issues to look out for:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"1. Memory Leak Risk (OOM Crashes)","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"1. Memory Leak Risk (OOM Crashes)","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Restart Logic:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The most significant change is the removal of the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Dependency Leaks:","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.050033245,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If any underlying service (like the","depth":26,"bounds":{"left":0.17453457,"top":0.0,"width":0.08277926,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ElasticSearchClient","depth":27,"bounds":{"left":0.25930852,"top":0.0,"width":0.056848403,"height":0.015163607},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", ORM, or Datadog helper) retains references to processed entities,","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.21775267,"height":0.03631285},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"bounds":{"left":0.26811835,"top":0.0,"width":0.056848403,"height":0.015163607},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.23138298,"height":0.05546688},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Hardcoded Limits:","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.047041222,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The code still includes","depth":26,"bounds":{"left":0.17154256,"top":0.0,"width":0.05651596,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ini_set('memory_limit', '512M');","depth":27,"bounds":{"left":0.23005319,"top":0.0,"width":0.095744684,"height":0.015163607},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":". Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.23055187,"height":0.05546688},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"2. Race Condition in Redis Lock","depth":23,"bounds":{"left":0.111369684,"top":0.07940942,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"2. Race Condition in Redis Lock","depth":24,"bounds":{"left":0.111369684,"top":0.079010375,"width":0.09125665,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Non-Atomic Operations:","depth":26,"bounds":{"left":0.12450133,"top":0.10614525,"width":0.062333778,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The","depth":26,"bounds":{"left":0.18683511,"top":0.10614525,"width":0.011635638,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage","depth":27,"bounds":{"left":0.20046543,"top":0.10774142,"width":0.041888297,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method uses three distinct steps to handle the lock:","depth":26,"bounds":{"left":0.12450133,"top":0.10614525,"width":0.22722739,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"get()","depth":27,"bounds":{"left":0.14860372,"top":0.12689546,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":",","depth":26,"bounds":{"left":0.16555852,"top":0.12529927,"width":0.002493351,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"bounds":{"left":0.17004654,"top":0.12689546,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", and","depth":26,"bounds":{"left":0.18700133,"top":0.12529927,"width":0.013131649,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"expire()","depth":27,"bounds":{"left":0.20212767,"top":0.12689546,"width":0.023936171,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"bounds":{"left":0.22805852,"top":0.12529927,"width":0.0013297872,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"bounds":{"left":0.12450133,"top":0.15403032,"width":0.040059842,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If multiple workers process the same","depth":26,"bounds":{"left":0.16456117,"top":0.15403032,"width":0.09275266,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$this->updateTarget","depth":27,"bounds":{"left":0.25930852,"top":0.15562649,"width":0.056848403,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"concurrently, they can both pass the","depth":26,"bounds":{"left":0.12450133,"top":0.15403032,"width":0.22855718,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"if (! is_null(Redis::get(...)))","depth":27,"bounds":{"left":0.18351063,"top":0.17478053,"width":0.09275266,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"check at the exact same millisecond. Both will then execute","depth":26,"bounds":{"left":0.12450133,"top":0.17318435,"width":0.21575798,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"bounds":{"left":0.2122673,"top":0.19393456,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and push duplicate metrics to Datadog.","depth":26,"bounds":{"left":0.22922207,"top":0.19233839,"width":0.09823803,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Fix:","depth":26,"bounds":{"left":0.12450133,"top":0.22106944,"width":0.019281914,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"This should be replaced with a single atomic operation using Redis's","depth":26,"bounds":{"left":0.14378324,"top":0.22106944,"width":0.16821809,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"NX","depth":27,"bounds":{"left":0.31399602,"top":0.22266561,"width":0.005984043,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Not eXists) and","depth":26,"bounds":{"left":0.12450133,"top":0.22106944,"width":0.22855718,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"EX","depth":27,"bounds":{"left":0.13696809,"top":0.24181964,"width":0.005984043,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Expire) flags, or Laravel’s built-in atomic cache locks:","depth":26,"bounds":{"left":0.14494681,"top":0.24022347,"width":0.13264628,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis::set($gaugeLockName, true, 'EX', 60, 'NX')","depth":27,"bounds":{"left":0.12649602,"top":0.26097366,"width":0.14361702,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"bounds":{"left":0.27210772,"top":0.25937748,"width":0.0013297872,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"3. Infinite Loops and Caller Contract","depth":23,"bounds":{"left":0.111369684,"top":0.30925778,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"3. Infinite Loops and Caller Contract","depth":24,"bounds":{"left":0.111369684,"top":0.30885875,"width":0.10571808,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Return Value Change:","depth":26,"bounds":{"left":0.12450133,"top":0.33599362,"width":0.055352394,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"bounds":{"left":0.18301196,"top":0.33758977,"width":0.06881649,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"previously returned whether it should iterate again based on staleness or chunk size. It now always returns","depth":26,"bounds":{"left":0.12450133,"top":0.33599362,"width":0.2237367,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"bounds":{"left":0.29438165,"top":0.3567438,"width":0.011968086,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(unless the selection is totally empty).","depth":26,"bounds":{"left":0.12450133,"top":0.35514766,"width":0.2130984,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"bounds":{"left":0.12450133,"top":0.40303272,"width":0.040059842,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the code calling this method relies on a","depth":26,"bounds":{"left":0.16456117,"top":0.40303272,"width":0.10305851,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"false","depth":27,"bounds":{"left":0.26961437,"top":0.4046289,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"return to pause, sleep, or gracefully terminate, changing this to an unconditional","depth":26,"bounds":{"left":0.12450133,"top":0.40303272,"width":0.22589761,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"bounds":{"left":0.26163563,"top":0.4237829,"width":0.011968086,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"could result in an aggressive \"busy loop\" that spikes CPU usage.","depth":26,"bounds":{"left":0.12450133,"top":0.42218676,"width":0.22290559,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"4. Performance Overhead","depth":23,"bounds":{"left":0.111369684,"top":0.49122107,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"4. Performance Overhead","depth":24,"bounds":{"left":0.111369684,"top":0.49082202,"width":0.076130316,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Expensive Garbage Collection:","depth":26,"bounds":{"left":0.12450133,"top":0.5179569,"width":0.078457445,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Calling","depth":26,"bounds":{"left":0.20295878,"top":0.5179569,"width":0.019281914,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles();","depth":27,"bounds":{"left":0.22423537,"top":0.51955307,"width":0.059840426,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"on","depth":26,"bounds":{"left":0.28607047,"top":0.5179569,"width":0.00880984,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"every single chunk","depth":26,"bounds":{"left":0.29488033,"top":0.5179569,"width":0.045545213,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.","depth":26,"bounds":{"left":0.12450133,"top":0.5179569,"width":0.23420878,"height":0.05546688},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Synchronous Metric Dispatch:","depth":26,"bounds":{"left":0.12450133,"top":0.584996,"width":0.07712766,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the","depth":26,"bounds":{"left":0.20162898,"top":0.584996,"width":0.015292553,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::increment","depth":27,"bounds":{"left":0.21891622,"top":0.5865922,"width":0.053856384,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"or","depth":26,"bounds":{"left":0.27476728,"top":0.584996,"width":0.0078125,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::gauge","depth":27,"bounds":{"left":0.28457448,"top":0.5865922,"width":0.041888297,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.","depth":26,"bounds":{"left":0.12450133,"top":0.584996,"width":0.23420878,"height":0.05546688},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"5. Lack of Explicit Error Handling","depth":23,"bounds":{"left":0.111369684,"top":0.67318434,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5. Lack of Explicit Error Handling","depth":24,"bounds":{"left":0.111369684,"top":0.67278534,"width":0.094913565,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"No","depth":26,"bounds":{"left":0.12450133,"top":0.6999202,"width":0.008477394,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"try/catch","depth":27,"bounds":{"left":0.1349734,"top":0.7015164,"width":0.026928192,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Blocks:","depth":26,"bounds":{"left":0.16389628,"top":0.6999202,"width":0.019614361,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The script executes complex network operations (","depth":26,"bounds":{"left":0.18351063,"top":0.6999202,"width":0.12367021,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"loadDocuments","depth":27,"bounds":{"left":0.30917552,"top":0.7015164,"width":0.038896278,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.","depth":26,"bounds":{"left":0.12450133,"top":0.6999202,"width":0.22888963,"height":0.09377494},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":22,"bounds":{"left":0.1087101,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":22,"bounds":{"left":0.124667555,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Redo","depth":22,"bounds":{"left":0.140625,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":21,"bounds":{"left":0.15658244,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":22,"bounds":{"left":0.17253989,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":21,"bounds":{"left":0.18849733,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXTextArea","text":"Ask Gemini","depth":21,"bounds":{"left":0.14594415,"top":0.86632085,"width":0.17021276,"height":0.01915403},"on_screen":true,"value":"Ask Gemini","help_text":"","role_description":"text entry area","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Ask Gemini","depth":22,"bounds":{"left":0.14594415,"top":0.8667199,"width":0.030418882,"height":0.018355945},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Upload & tools","depth":20,"bounds":{"left":0.12732713,"top":0.8567438,"width":0.015957447,"height":0.03830806},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Microphone","depth":20,"bounds":{"left":0.32413563,"top":0.86153233,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Send message","depth":20,"bounds":{"left":0.33211437,"top":0.8631285,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":false,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.","depth":17,"bounds":{"left":0.102726065,"top":0.9173983,"width":0.21775267,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Your privacy & Gemini Opens in a new window","depth":17,"bounds":{"left":0.32047874,"top":0.9173983,"width":0.043882977,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Your privacy & Gemini","depth":18,"bounds":{"left":0.32047874,"top":0.9173983,"width":0.043882977,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Opens in a new window","depth":19,"bounds":{"left":0.079288565,"top":0.91660017,"width":0.04720745,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Summarize page","depth":7,"bounds":{"left":0.08494016,"top":0.95730245,"width":0.053523935,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Summarize page","depth":9,"bounds":{"left":0.09059176,"top":0.96249,"width":0.042220745,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Skip to content","depth":6,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Skip to content","depth":7,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Open menu","depth":10,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"Homepage (g then d)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"jiminny","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"jiminny","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"app","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"app","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Search or jump to…","depth":9,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Type","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"/","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to search","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Chat with Copilot","depth":10,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXMenuButton","text":"Open Copilot…","depth":9,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXMenuButton","text":"Create new...","depth":9,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"All issues(g then i)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"All pull requests","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"All repositories","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"You have unread notifications(g then n)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Open user navigation menu","depth":9,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Repository navigation","depth":9,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Repository navigation","depth":10,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Code","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Code","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Pull requests (32)","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Pull requests","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"32","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Agents","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Agents","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Actions","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Actions","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Wiki","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Wiki","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Security and quality","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Security and quality","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Insights","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Insights","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Settings","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Settings","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"JY-20915 add alias for EU #12105 Edit title","depth":13,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"JY-20915 add alias for EU","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"#","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"12105","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Edit title","depth":14,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Awaiting approval","depth":13,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Awaiting approval","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXMenuButton","text":"Code","depth":13,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXStaticText","text":"Code","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Open","depth":13,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"LakyLak","depth":15,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"wants to merge 5 commits into","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"master","depth":15,"on_screen":false,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"master","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"from","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915-add-domain-specific-email-text-relay","depth":16,"on_screen":false,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915-add-domain-specific-email-text-relay","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy head branch name to clipboard","depth":16,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Lines changed: 365 additions & 1 deletion","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Conversation (1)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"Conversation","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"1","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Commits (5)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Commits","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Checks (3)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Checks","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"3","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Files changed (4)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Files changed","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"4","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Open","depth":14,"bounds":{"left":0.50332445,"top":0.07222666,"width":0.011968086,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"JY-20915 add alias for EU #12105 LakyLak wants to merge 5 commits into master from JY-20915-add-domain-specific-email-text-relay Copy head branch name to clipboard","depth":14,"bounds":{"left":0.5219415,"top":0.058260176,"width":0.22922207,"height":0.042298485},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915 add alias for EU","depth":16,"bounds":{"left":0.5219415,"top":0.05865922,"width":0.05817819,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915 add alias for EU","depth":17,"bounds":{"left":0.5219415,"top":0.06304868,"width":0.05817819,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"#","depth":16,"bounds":{"left":0.5827792,"top":0.06304868,"width":0.0028257978,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"12105","depth":16,"bounds":{"left":0.585605,"top":0.06304868,"width":0.012799202,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"LakyLak","depth":18,"bounds":{"left":0.5219415,"top":0.08339984,"width":0.016123671,"height":0.011971269},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":19,"bounds":{"left":0.5219415,"top":0.08339984,"width":0.016123671,"height":0.011971269},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"wants to merge 5 commits into","depth":18,"bounds":{"left":0.539395,"top":0.08339984,"width":0.058011968,"height":0.011971269},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"master","depth":18,"bounds":{"left":0.5987367,"top":0.08180367,"width":0.018450798,"height":0.015163607},"on_screen":true,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"master","depth":19,"bounds":{"left":0.6007314,"top":0.08339984,"width":0.014461436,"height":0.011572227},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"from","depth":19,"bounds":{"left":0.6185173,"top":0.08339984,"width":0.008643617,"height":0.011971269},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915-add-domain-specific-email-text-relay","depth":19,"bounds":{"left":0.6284907,"top":0.08180367,"width":0.112034574,"height":0.015163607},"on_screen":true,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915-add-domain-specific-email-text-relay","depth":20,"bounds":{"left":0.63048536,"top":0.08339984,"width":0.10804521,"height":0.011572227},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy head branch name to clipboard","depth":19,"bounds":{"left":0.741855,"top":0.07821229,"width":0.00930851,"height":0.022346368},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Conversation","depth":12,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Conversation","depth":13,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@LakyLak","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show options","depth":15,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXHeading","text":"LakyLak commented yesterday","depth":14,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXLink","text":"LakyLak","depth":16,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"commented","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"yesterday","depth":15,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"yesterday","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"JIRA: JY-20915","depth":16,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"JIRA:","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915","depth":17,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Changes:","depth":16,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Changes:","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Support text relay for EU","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Prevent errors from EU on US","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Add or remove reactions","depth":16,"on_screen":false,"help_text":"","role_description":"summary","subrole":"AXSummary","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"LakyLak","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"added","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"commits","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"yesterday","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"yesterday","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@LakyLak","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"JY-20915","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"add alias for EU","depth":14,"on_screen":false,"help_text":"JY-20915 add alias for EU","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"add alias for EU","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"8 / 10 checks OK","depth":14,"on_screen":false,"help_text":"","role_description":"summary","subrole":"AXSummary","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"e5255d5","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"e5255d5","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@LakyLak","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"JY-20915","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"fix env recognization","depth":14,"on_screen":false,"help_text":"JY-20915 fix env recognization","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"fix env recognization","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"11 / 12 checks OK","depth":14,"on_screen":false,"help_text":"","role_description":"summary","subrole":"AXSummary","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"8953054","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"8953054","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@LakyLak","depth":12,"bounds":{"left":0.5249335,"top":0.0,"width":0.0066489363,"height":0.015961692},"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"JY-20915","depth":14,"bounds":{"left":0.53357714,"top":0.0,"width":0.019115692,"height":0.011572227},"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915","depth":15,"bounds":{"left":0.53357714,"top":0.0,"width":0.019115692,"height":0.011572227},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"code smells & tests","depth":14,"bounds":{"left":0.55502,"top":0.0,"width":0.045711435,"height":0.011572227},"on_screen":false,"help_text":"JY-20915 code smells & tests","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"code smells & tests","depth":15,"bounds":{"left":0.55502,"top":0.0,"width":0.045711435,"height":0.011572227},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"11 / 12 checks OK","depth":14,"bounds":{"left":0.7563165,"top":0.0,"width":0.005319149,"height":0.016759777},"on_screen":false,"help_text":"","role_description":"summary","subrole":"AXSummary","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"73d4bea","depth":14,"bounds":{"left":0.76296544,"top":0.0,"width":0.016954787,"height":0.011572227},"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"73d4bea","depth":15,"bounds":{"left":0.76296544,"top":0.0,"width":0.016954787,"height":0.011572227},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@LakyLak","depth":12,"bounds":{"left":0.5249335,"top":0.0,"width":0.0066489363,"height":0.015961692},"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"JY-20915","depth":14,"bounds":{"left":0.53357714,"top":0.0,"width":0.019115692,"height":0.011572227},"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915","depth":15,"bounds":{"left":0.53357714,"top":0.0,"width":0.019115692,"height":0.011572227},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"add more tests","depth":14,"bounds":{"left":0.55502,"top":0.0,"width":0.03357713,"height":0.011572227},"on_screen":false,"help_text":"JY-20915 add more tests","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"add more tests","depth":15,"bounds":{"left":0.55502,"top":0.0,"width":0.03357713,"height":0.011572227},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"11 / 12 checks OK","depth":14,"bounds":{"left":0.7563165,"top":0.0,"width":0.005319149,"height":0.016759777},"on_screen":false,"help_text":"","role_description":"summary","subrole":"AXSummary","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"5d2dd3e","depth":14,"bounds":{"left":0.76296544,"top":0.0,"width":0.016954787,"height":0.011572227},"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"5d2dd3e","depth":15,"bounds":{"left":0.76296544,"top":0.0,"width":0.016954787,"height":0.011572227},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@LakyLak","depth":12,"bounds":{"left":0.5249335,"top":0.0,"width":0.0066489363,"height":0.015961692},"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"JY-20915","depth":14,"bounds":{"left":0.53357714,"top":0.0,"width":0.019115692,"height":0.011572227},"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915","depth":15,"bounds":{"left":0.53357714,"top":0.0,"width":0.019115692,"height":0.011572227},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"add sms received test","depth":14,"bounds":{"left":0.55502,"top":0.0,"width":0.050531916,"height":0.011572227},"on_screen":false,"help_text":"JY-20915 add sms received test","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"add sms received test","depth":15,"bounds":{"left":0.55502,"top":0.0,"width":0.050531916,"height":0.011572227},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"12 / 12 checks OK","depth":14,"bounds":{"left":0.7563165,"top":0.0,"width":0.005319149,"height":0.016759777},"on_screen":false,"help_text":"","role_description":"summary","subrole":"AXSummary","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"b8c8eb5","depth":14,"bounds":{"left":0.76296544,"top":0.0,"width":0.016954787,"height":0.011572227},"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"b8c8eb5","depth":15,"bounds":{"left":0.76296544,"top":0.0,"width":0.016954787,"height":0.011572227},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@sonarqubecloud","depth":13,"bounds":{"left":0.49268618,"top":0.019952115,"width":0.013297873,"height":0.031923383},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show options","depth":14,"bounds":{"left":0.7662899,"top":0.0207502,"width":0.007978723,"height":0.02952913},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXHeading","text":"sonarqubecloud Bot commented 3 hours ago","depth":13,"bounds":{"left":0.5169548,"top":0.0207502,"width":0.24135639,"height":0.029928172},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXLink","text":"sonarqubecloud","depth":15,"bounds":{"left":0.5169548,"top":0.028731046,"width":0.036236703,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"sonarqubecloud","depth":16,"bounds":{"left":0.5169548,"top":0.028731046,"width":0.036236703,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Bot","depth":16,"bounds":{"left":0.55668217,"top":0.030327214,"width":0.0066489363,"height":0.011971269},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"commented","depth":14,"bounds":{"left":0.56698805,"top":0.029130088,"width":0.025598405,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"3 hours ago","depth":14,"bounds":{"left":0.59391624,"top":0.027533919,"width":0.025598405,"height":0.016759777},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"3 hours ago","depth":16,"bounds":{"left":0.59391624,"top":0.029130088,"width":0.025598405,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Quality Gate Passed Quality Gate passed","depth":16,"bounds":{"left":0.5169548,"top":0.06424581,"width":0.25731382,"height":0.026735835},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXLink","text":"Quality Gate Passed","depth":17,"bounds":{"left":0.5169548,"top":0.0650439,"width":0.0066489363,"height":0.019952115},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Quality Gate passed","depth":18,"bounds":{"left":0.52526593,"top":0.0650439,"width":0.06416223,"height":0.019952115},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Issues","depth":17,"bounds":{"left":0.5169548,"top":0.105347164,"width":0.013464096,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"0 New issues","depth":17,"bounds":{"left":0.5234375,"top":0.12210695,"width":0.028590426,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"0 New issues","depth":18,"bounds":{"left":0.5234375,"top":0.12210695,"width":0.028590426,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"0 Accepted issues","depth":17,"bounds":{"left":0.5234375,"top":0.13886672,"width":0.03956117,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"0 Accepted issues","depth":18,"bounds":{"left":0.5234375,"top":0.13886672,"width":0.03956117,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Measures","depth":17,"bounds":{"left":0.5169548,"top":0.16839585,"width":0.020777926,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"0 Security Hotspots","depth":17,"bounds":{"left":0.5234375,"top":0.18515563,"width":0.04288564,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"0 Security Hotspots","depth":18,"bounds":{"left":0.5234375,"top":0.18515563,"width":0.04288564,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"72.2% Coverage on New Code","depth":17,"bounds":{"left":0.5234375,"top":0.2019154,"width":0.06565824,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"72.2% Coverage on New Code","depth":18,"bounds":{"left":0.5234375,"top":0.2019154,"width":0.06565824,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"0.0% Duplication on New Code","depth":17,"bounds":{"left":0.5234375,"top":0.21867518,"width":0.066821806,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"0.0% Duplication on New Code","depth":18,"bounds":{"left":0.5234375,"top":0.21867518,"width":0.066821806,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"}]...
|
378147860039457543
|
-1074328845390940392
|
idle
|
accessibility
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo
Share & export
Copy
Show more options
Ask Gemini
Ask Gemini
Upload & tools
Microphone
Send message
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Summarize page
Summarize page
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
All issues(g then i)
All pull requests
All repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Pull requests (32)
Pull requests
(
32
)
Agents
Agents
Actions
Actions
Wiki
Wiki
Security and quality
Security and quality
Insights
Insights
Settings
Settings
JY-20915 add alias for EU #12105 Edit title
JY-20915 add alias for EU
#
12105
Edit title
Awaiting approval
Awaiting approval
Code
Code
Open
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Lines changed: 365 additions & 1 deletion
Conversation (1)
Conversation
(
1
)
Commits (5)
Commits
(
5
)
Checks (3)
Checks
(
3
)
Files changed (4)
Files changed
(
4
)
Open
JY-20915 add alias for EU #12105 LakyLak wants to merge 5 commits into master from JY-20915-add-domain-specific-email-text-relay Copy head branch name to clipboard
JY-20915 add alias for EU
JY-20915 add alias for EU
#
12105
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Conversation
Conversation
@LakyLak
Show options
LakyLak commented yesterday
LakyLak
LakyLak
commented
yesterday
yesterday
JIRA: JY-20915
JIRA:
JY-20915
JY-20915
Changes:
Changes:
Support text relay for EU
Prevent errors from EU on US
Add or remove reactions
LakyLak
LakyLak
added
5
commits
yesterday
yesterday
@LakyLak
JY-20915
JY-20915
add alias for EU
add alias for EU
8 / 10 checks OK
e5255d5
e5255d5
@LakyLak
JY-20915
JY-20915
fix env recognization
fix env recognization
11 / 12 checks OK
8953054
8953054
@LakyLak
JY-20915
JY-20915
code smells & tests
code smells & tests
11 / 12 checks OK
73d4bea
73d4bea
@LakyLak
JY-20915
JY-20915
add more tests
add more tests
11 / 12 checks OK
5d2dd3e
5d2dd3e
@LakyLak
JY-20915
JY-20915
add sms received test
add sms received test
12 / 12 checks OK
b8c8eb5
b8c8eb5
@sonarqubecloud
Show options
sonarqubecloud Bot commented 3 hours ago
sonarqubecloud
sonarqubecloud
Bot
commented
3 hours ago
3 hours ago
Quality Gate Passed Quality Gate passed
Quality Gate Passed
Quality Gate passed
Issues
0 New issues
0 New issues
0 Accepted issues
0 Accepted issues
Measures
0 Security Hotspots
0 Security Hotspots
72.2% Coverage on New Code
72.2% Coverage on New Code
0.0% Duplication on New Code
0.0% Duplication on New Code...
|
65234
|
NULL
|
NULL
|
NULL
|
|
65237
|
2287
|
2
|
2026-05-20T15:31:17.564499+00:00
|
/Users/lukas/.screenpipe/data/data/2026-05-20/1779 /Users/lukas/.screenpipe/data/data/2026-05-20/1779291077564_m1.jpg...
|
Firefox
|
JY-20915 add alias for EU by LakyLak · Pull Reques JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app — Work...
|
1
|
github.com/jiminny/app/pull/12105
|
monitor_1
|
NULL
|
NULL
|
NULL
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo
Share & export
Copy
Show more options
Ask Gemini
Ask Gemini
Upload & tools
Microphone
Send message
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Summarize page
Summarize page
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
All issues(g then i)
All pull requests
All repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Pull requests (32)
Pull requests
(
32
)
Agents
Agents
Actions
Actions
Wiki
Wiki
Security and quality
Security and quality
Insights
Insights
Settings
Settings
JY-20915 add alias for EU #12105 Edit title
JY-20915 add alias for EU
#
12105
Edit title
Awaiting approval
Awaiting approval
Code
Code
Open
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Lines changed: 365 additions & 1 deletion
Conversation (1)
Conversation
(
1
)
Commits (5)
Commits
(
5
)
Checks (3)
Checks
(
3
)
Files changed (4)
Files changed
(
4
)
Open
JY-20915 add alias for EU #12105 LakyLak wants to merge 5 commits into master from JY-20915-add-domain-specific-email-text-relay Copy head branch name to clipboard
JY-20915 add alias for EU
JY-20915 add alias for EU
#
12105
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Conversation
Conversation
@LakyLak
Show options
LakyLak commented yesterday
LakyLak
LakyLak
commented
yesterday
yesterday
JIRA: JY-20915
JIRA:
JY-20915
JY-20915
Changes:
Changes:
Support text relay for EU
Prevent errors from EU on US
Add or remove reactions
LakyLak
LakyLak
added
5
commits
yesterday
yesterday
@LakyLak
JY-20915
JY-20915
add alias for EU
add alias for EU
8 / 10 checks OK
e5255d5
e5255d5
@LakyLak
JY-20915
JY-20915
fix env recognization
fix env recognization
11 / 12 checks OK
8953054
8953054
@LakyLak
JY-20915
JY-20915
code smells & tests
code smells & tests
11 / 12 checks OK
73d4bea
73d4bea
@LakyLak...
|
[{"role":"AXRadioButton","text [{"role":"AXRadioButton","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Close tab","depth":5,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXRadioButton","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Pipelines - jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Pipelines - jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Tab","depth":4,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Customize sidebar","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Close Google Gemini (⌃X)","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Tabs from other devices","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open history (⇧⌘H)","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open bookmarks (⌘B)","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"AI Chat settings","depth":7,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Close","depth":7,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Main menu","depth":13,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"New chat","depth":12,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Gemini","depth":15,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Chat","depth":13,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Open menu for conversation actions.","depth":13,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Conversation with Gemini","depth":15,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Conversation with Gemini","depth":16,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"You said I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.","depth":23,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Expand","depth":21,"on_screen":false,"role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a concise summary of the changes made in the code selection:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Refactoring and Dependencies","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Refactoring and Dependencies","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Replaced Interface with Implementation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Changed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusInterface","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusManager","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"in the imports and constructor dependency injection.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Redis Support:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Imported","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Illuminate\\Support\\Facades\\Redis","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to handle a newly introduced memory-logging lock mechanism.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Traits:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"WorkerIdTrait","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", along with the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"worker","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"field tracking in the final logs.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Loop and Worker Lifecycle Adjustments","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Loop and Worker Lifecycle Adjustments","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Worker Auto-Restart:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Deleted the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property, the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"constant, and the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"shouldIterateAgain()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method. The script no longer tracks execution iterations or triggers a self-managed worker restart.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Simplified Chunk Method Return:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Modified","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to always return","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Priority Handling for Throttled Entities","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Priority Handling for Throttled Entities","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Stale Record Elevation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Updated the logic in","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"rescheduleThrottledEntities()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to check if a throttled entity group is classified as \"stale\". If it is, the entities are rescheduled using","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"QueuePriorityEnum::NORMAL","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"; otherwise, they keep their respective queue priority.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Metrics, Logging, and Environment Handling","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Metrics, Logging, and Environment Handling","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Environment Tracking:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Introduced a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$environment","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property (defaulting to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"'local'","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") and a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"setEnvironment()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method to dynamically assign it.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Memory Monitoring via Datadog Gauge:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method that submits peak memory allocation to Datadog as a gauge metric (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"jiminny.reindex-memory-usage","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis Lock for Metrics Throttling:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Implemented a Redis-backed lock (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"%s-process-manager-lock","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") with a 60-second expiration inside","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to prevent over-reporting metrics.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Log Payload Update:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added total processed items (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$updatedDocuments + $deletedDocuments","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") to the final information log.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":23,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"You said are there any issues or potential bottlenecks","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"are there any issues or potential bottlenecks","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Edit","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a breakdown of the potential issues to look out for:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"1. Memory Leak Risk (OOM Crashes)","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"1. Memory Leak Risk (OOM Crashes)","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Restart Logic:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The most significant change is the removal of the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Dependency Leaks:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If any underlying service (like the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ElasticSearchClient","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", ORM, or Datadog helper) retains references to processed entities,","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Hardcoded Limits:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The code still includes","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ini_set('memory_limit', '512M');","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":". Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"2. Race Condition in Redis Lock","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"2. Race Condition in Redis Lock","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Non-Atomic Operations:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method uses three distinct steps to handle the lock:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"get()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":",","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", and","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"expire()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If multiple workers process the same","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$this->updateTarget","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"concurrently, they can both pass the","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"if (! is_null(Redis::get(...)))","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"check at the exact same millisecond. Both will then execute","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and push duplicate metrics to Datadog.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Fix:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"This should be replaced with a single atomic operation using Redis's","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"NX","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Not eXists) and","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"EX","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Expire) flags, or Laravel’s built-in atomic cache locks:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis::set($gaugeLockName, true, 'EX', 60, 'NX')","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"3. Infinite Loops and Caller Contract","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"3. Infinite Loops and Caller Contract","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Return Value Change:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"previously returned whether it should iterate again based on staleness or chunk size. It now always returns","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(unless the selection is totally empty).","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the code calling this method relies on a","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"false","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"return to pause, sleep, or gracefully terminate, changing this to an unconditional","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"could result in an aggressive \"busy loop\" that spikes CPU usage.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"4. Performance Overhead","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"4. Performance Overhead","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Expensive Garbage Collection:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Calling","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles();","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"on","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"every single chunk","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Synchronous Metric Dispatch:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::increment","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"or","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::gauge","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"5. Lack of Explicit Error Handling","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5. Lack of Explicit Error Handling","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"No","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"try/catch","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Blocks:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The script executes complex network operations (","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"loadDocuments","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":22,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":22,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Redo","depth":22,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":21,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":22,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":21,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXTextArea","text":"Ask Gemini","depth":21,"on_screen":true,"value":"Ask Gemini","help_text":"","role_description":"text entry area","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Ask Gemini","depth":22,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Upload & tools","depth":20,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Microphone","depth":20,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Send message","depth":20,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":false,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.","depth":17,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Your privacy & Gemini Opens in a new window","depth":17,"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Your privacy & Gemini","depth":18,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Opens in a new window","depth":19,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Summarize page","depth":7,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Summarize page","depth":9,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Skip to content","depth":6,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Skip to content","depth":7,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Open menu","depth":10,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"Homepage (g then d)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"jiminny","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"jiminny","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"app","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"app","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Search or jump to…","depth":9,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Type","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"/","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to search","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Chat with Copilot","depth":10,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXMenuButton","text":"Open Copilot…","depth":9,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXMenuButton","text":"Create new...","depth":9,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"All issues(g then i)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"All pull requests","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"All repositories","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"You have unread notifications(g then n)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Open user navigation menu","depth":9,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Repository navigation","depth":9,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Repository navigation","depth":10,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Code","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Code","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Pull requests (32)","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Pull requests","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"32","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Agents","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Agents","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Actions","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Actions","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Wiki","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Wiki","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Security and quality","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Security and quality","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Insights","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Insights","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Settings","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Settings","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"JY-20915 add alias for EU #12105 Edit title","depth":13,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"JY-20915 add alias for EU","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"#","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"12105","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Edit title","depth":14,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Awaiting approval","depth":13,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Awaiting approval","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXMenuButton","text":"Code","depth":13,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXStaticText","text":"Code","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Open","depth":13,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"LakyLak","depth":15,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"wants to merge 5 commits into","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"master","depth":15,"on_screen":false,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"master","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"from","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915-add-domain-specific-email-text-relay","depth":16,"on_screen":false,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915-add-domain-specific-email-text-relay","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy head branch name to clipboard","depth":16,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Lines changed: 365 additions & 1 deletion","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Conversation (1)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"Conversation","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"1","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Commits (5)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Commits","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Checks (3)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Checks","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"3","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Files changed (4)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Files changed","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"4","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Open","depth":14,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"JY-20915 add alias for EU #12105 LakyLak wants to merge 5 commits into master from JY-20915-add-domain-specific-email-text-relay Copy head branch name to clipboard","depth":14,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915 add alias for EU","depth":16,"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915 add alias for EU","depth":17,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"#","depth":16,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"12105","depth":16,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"LakyLak","depth":18,"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":19,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"wants to merge 5 commits into","depth":18,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"master","depth":18,"on_screen":true,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"master","depth":19,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"from","depth":19,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915-add-domain-specific-email-text-relay","depth":19,"on_screen":true,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915-add-domain-specific-email-text-relay","depth":20,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy head branch name to clipboard","depth":19,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Conversation","depth":12,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Conversation","depth":13,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@LakyLak","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show options","depth":15,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXHeading","text":"LakyLak commented yesterday","depth":14,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXLink","text":"LakyLak","depth":16,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"commented","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"yesterday","depth":15,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"yesterday","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"JIRA: JY-20915","depth":16,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"JIRA:","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915","depth":17,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Changes:","depth":16,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Changes:","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Support text relay for EU","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Prevent errors from EU on US","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Add or remove reactions","depth":16,"on_screen":false,"help_text":"","role_description":"summary","subrole":"AXSummary","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"LakyLak","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"added","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"commits","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"yesterday","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"yesterday","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@LakyLak","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"JY-20915","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"add alias for EU","depth":14,"on_screen":false,"help_text":"JY-20915 add alias for EU","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"add alias for EU","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"8 / 10 checks OK","depth":14,"on_screen":false,"help_text":"","role_description":"summary","subrole":"AXSummary","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"e5255d5","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"e5255d5","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@LakyLak","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"JY-20915","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"fix env recognization","depth":14,"on_screen":false,"help_text":"JY-20915 fix env recognization","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"fix env recognization","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"11 / 12 checks OK","depth":14,"on_screen":false,"help_text":"","role_description":"summary","subrole":"AXSummary","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"8953054","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"8953054","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@LakyLak","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"JY-20915","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"code smells & tests","depth":14,"on_screen":false,"help_text":"JY-20915 code smells & tests","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"code smells & tests","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"11 / 12 checks OK","depth":14,"on_screen":false,"help_text":"","role_description":"summary","subrole":"AXSummary","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"73d4bea","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"73d4bea","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@LakyLak","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false}]...
|
-7372317613308033634
|
-1048293509551605992
|
idle
|
accessibility
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo
Share & export
Copy
Show more options
Ask Gemini
Ask Gemini
Upload & tools
Microphone
Send message
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Summarize page
Summarize page
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
All issues(g then i)
All pull requests
All repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Pull requests (32)
Pull requests
(
32
)
Agents
Agents
Actions
Actions
Wiki
Wiki
Security and quality
Security and quality
Insights
Insights
Settings
Settings
JY-20915 add alias for EU #12105 Edit title
JY-20915 add alias for EU
#
12105
Edit title
Awaiting approval
Awaiting approval
Code
Code
Open
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Lines changed: 365 additions & 1 deletion
Conversation (1)
Conversation
(
1
)
Commits (5)
Commits
(
5
)
Checks (3)
Checks
(
3
)
Files changed (4)
Files changed
(
4
)
Open
JY-20915 add alias for EU #12105 LakyLak wants to merge 5 commits into master from JY-20915-add-domain-specific-email-text-relay Copy head branch name to clipboard
JY-20915 add alias for EU
JY-20915 add alias for EU
#
12105
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Conversation
Conversation
@LakyLak
Show options
LakyLak commented yesterday
LakyLak
LakyLak
commented
yesterday
yesterday
JIRA: JY-20915
JIRA:
JY-20915
JY-20915
Changes:
Changes:
Support text relay for EU
Prevent errors from EU on US
Add or remove reactions
LakyLak
LakyLak
added
5
commits
yesterday
yesterday
@LakyLak
JY-20915
JY-20915
add alias for EU
add alias for EU
8 / 10 checks OK
e5255d5
e5255d5
@LakyLak
JY-20915
JY-20915
fix env recognization
fix env recognization
11 / 12 checks OK
8953054
8953054
@LakyLak
JY-20915
JY-20915
code smells & tests
code smells & tests
11 / 12 checks OK
73d4bea
73d4bea
@LakyLak...
|
65235
|
NULL
|
NULL
|
NULL
|
|
65238
|
2288
|
2
|
2026-05-20T15:31:29.064619+00:00
|
/Users/lukas/.screenpipe/data/data/2026-05-20/1779 /Users/lukas/.screenpipe/data/data/2026-05-20/1779291089064_m2.jpg...
|
Firefox
|
JY-20915 add alias for EU by LakyLak · Pull Reques JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app — Work...
|
1
|
github.com/jiminny/app/pull/12105
|
monitor_2
|
NULL
|
NULL
|
NULL
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo
Share & export
Copy
Show more options
Ask Gemini
Ask Gemini
Upload & tools
Microphone
Send message
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Summarize page
Summarize page
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
All issues(g then i)
All pull requests
All repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Pull requests (32)
Pull requests
(
32
)
Agents
Agents
Actions
Actions
Wiki
Wiki
Security and quality
Security and quality
Insights
Insights
Settings
Settings
JY-20915 add alias for EU #12105 Edit title
JY-20915 add alias for EU
#
12105
Edit title
Awaiting approval
Awaiting approval
Code
Code
Open
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Lines changed: 365 additions & 1 deletion
Conversation (1)
Conversation
(
1
)
Commits (5)
Commits
(
5
)
Checks (3)
Checks
(
3
)
Files changed (4)
Files changed
(
4
)
Open
JY-20915 add alias for EU #12105 LakyLak wants to merge 5 commits into master from JY-20915-add-domain-specific-email-text-relay Copy head branch name to clipboard
JY-20915 add alias for EU
JY-20915 add alias for EU
#
12105
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Conversation
Conversation
@LakyLak
Show options
LakyLak commented yesterday
LakyLak
LakyLak
commented
yesterday
yesterday
JIRA: JY-20915
JIRA:
JY-20915
JY-20915
Changes:
Changes:
Support text relay for EU
Prevent errors from EU on US
Add or remove reactions
LakyLak
LakyLak
added
5
commits
yesterday
yesterday
@LakyLak
JY-20915
JY-20915
add alias for EU
add alias for EU
8 / 10 checks OK
e5255d5
e5255d5
@LakyLak
JY-20915
JY-20915
fix env recognization
fix env recognization
11 / 12 checks OK
8953054
8953054
@LakyLak
JY-20915
JY-20915
code smells & tests
code smells & tests
11 / 12 checks OK
73d4bea
73d4bea
@LakyLak
JY-20915
JY-20915
add more tests
add more tests
11 / 12 checks OK
5d2dd3e
5d2dd3e
@LakyLak
JY-20915
JY-20915
add sms received test
add sms received test
12 / 12 checks OK
b8c8eb5
b8c8eb5
@sonarqubecloud
Show options
sonarqubecloud Bot commented 3 hours ago
sonarqubecloud
sonarqubecloud...
|
[{"role":"AXRadioButton","text [{"role":"AXRadioButton","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":4,"bounds":{"left":0.0,"top":0.0518755,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.06304868,"width":0.10106383,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":4,"bounds":{"left":0.0,"top":0.08459697,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.09577015,"width":0.10721409,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.11731844,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.12849163,"width":0.15791224,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.15003991,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.16121309,"width":0.1796875,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.18276137,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.19393456,"width":0.12699468,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Close tab","depth":5,"bounds":{"left":0.06732048,"top":0.18994413,"width":0.007978723,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXRadioButton","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":4,"bounds":{"left":0.0,"top":0.21548285,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":5,"bounds":{"left":0.013297873,"top":0.22665602,"width":0.10139628,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":4,"bounds":{"left":0.0,"top":0.2482043,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.25937748,"width":0.18068483,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":4,"bounds":{"left":0.0,"top":0.28092578,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":5,"bounds":{"left":0.013297873,"top":0.29209897,"width":0.4644282,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Pipelines - jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.31364724,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Pipelines - jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.32482043,"width":0.039228722,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.3463687,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.3575419,"width":0.15824468,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Tab","depth":4,"bounds":{"left":0.0028257978,"top":0.38068634,"width":0.07413564,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Customize sidebar","depth":6,"bounds":{"left":0.0028257978,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Close Google Gemini (⌃X)","depth":6,"bounds":{"left":0.013796543,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Tabs from other devices","depth":6,"bounds":{"left":0.024933511,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open history (⇧⌘H)","depth":6,"bounds":{"left":0.036070477,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open bookmarks (⌘B)","depth":6,"bounds":{"left":0.04720745,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"AI Chat settings","depth":7,"bounds":{"left":0.35854387,"top":0.055067837,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Close","depth":7,"bounds":{"left":0.37051198,"top":0.055067837,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Main menu","depth":13,"bounds":{"left":0.08228058,"top":0.09736632,"width":0.015957447,"height":0.03830806},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"New chat","depth":12,"bounds":{"left":0.09823803,"top":0.10215483,"width":0.028091755,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Gemini","depth":15,"bounds":{"left":0.1008976,"top":0.10654429,"width":0.02144282,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Chat","depth":13,"bounds":{"left":0.35854387,"top":0.10215483,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Open menu for conversation actions.","depth":13,"bounds":{"left":0.37051198,"top":0.10215483,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Conversation with Gemini","depth":15,"bounds":{"left":0.079288565,"top":0.14445332,"width":0.0003324468,"height":0.0007980846},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Conversation with Gemini","depth":16,"bounds":{"left":0.079288565,"top":0.14684756,"width":0.1200133,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"You said I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.","depth":23,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Expand","depth":21,"on_screen":false,"role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a concise summary of the changes made in the code selection:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Refactoring and Dependencies","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Refactoring and Dependencies","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Replaced Interface with Implementation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Changed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusInterface","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusManager","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"in the imports and constructor dependency injection.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Redis Support:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Imported","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Illuminate\\Support\\Facades\\Redis","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to handle a newly introduced memory-logging lock mechanism.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Traits:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"WorkerIdTrait","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", along with the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"worker","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"field tracking in the final logs.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Loop and Worker Lifecycle Adjustments","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Loop and Worker Lifecycle Adjustments","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Worker Auto-Restart:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Deleted the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property, the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"constant, and the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"shouldIterateAgain()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method. The script no longer tracks execution iterations or triggers a self-managed worker restart.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Simplified Chunk Method Return:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Modified","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to always return","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Priority Handling for Throttled Entities","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Priority Handling for Throttled Entities","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Stale Record Elevation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Updated the logic in","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"rescheduleThrottledEntities()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to check if a throttled entity group is classified as \"stale\". If it is, the entities are rescheduled using","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"QueuePriorityEnum::NORMAL","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"; otherwise, they keep their respective queue priority.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Metrics, Logging, and Environment Handling","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Metrics, Logging, and Environment Handling","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Environment Tracking:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Introduced a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$environment","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property (defaulting to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"'local'","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") and a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"setEnvironment()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method to dynamically assign it.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Memory Monitoring via Datadog Gauge:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method that submits peak memory allocation to Datadog as a gauge metric (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"jiminny.reindex-memory-usage","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis Lock for Metrics Throttling:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Implemented a Redis-backed lock (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"%s-process-manager-lock","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") with a 60-second expiration inside","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to prevent over-reporting metrics.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Log Payload Update:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added total processed items (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$updatedDocuments + $deletedDocuments","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") to the final information log.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":23,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"You said are there any issues or potential bottlenecks","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"are there any issues or potential bottlenecks","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Edit","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a breakdown of the potential issues to look out for:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"1. Memory Leak Risk (OOM Crashes)","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"1. Memory Leak Risk (OOM Crashes)","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Restart Logic:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The most significant change is the removal of the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Dependency Leaks:","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.050033245,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If any underlying service (like the","depth":26,"bounds":{"left":0.17453457,"top":0.0,"width":0.08277926,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ElasticSearchClient","depth":27,"bounds":{"left":0.25930852,"top":0.0,"width":0.056848403,"height":0.015163607},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", ORM, or Datadog helper) retains references to processed entities,","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.21775267,"height":0.03631285},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"bounds":{"left":0.26811835,"top":0.0,"width":0.056848403,"height":0.015163607},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.23138298,"height":0.05546688},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Hardcoded Limits:","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.047041222,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The code still includes","depth":26,"bounds":{"left":0.17154256,"top":0.0,"width":0.05651596,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ini_set('memory_limit', '512M');","depth":27,"bounds":{"left":0.23005319,"top":0.0,"width":0.095744684,"height":0.015163607},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":". Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.23055187,"height":0.05546688},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"2. Race Condition in Redis Lock","depth":23,"bounds":{"left":0.111369684,"top":0.07940942,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"2. Race Condition in Redis Lock","depth":24,"bounds":{"left":0.111369684,"top":0.079010375,"width":0.09125665,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Non-Atomic Operations:","depth":26,"bounds":{"left":0.12450133,"top":0.10614525,"width":0.062333778,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The","depth":26,"bounds":{"left":0.18683511,"top":0.10614525,"width":0.011635638,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage","depth":27,"bounds":{"left":0.20046543,"top":0.10774142,"width":0.041888297,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method uses three distinct steps to handle the lock:","depth":26,"bounds":{"left":0.12450133,"top":0.10614525,"width":0.22722739,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"get()","depth":27,"bounds":{"left":0.14860372,"top":0.12689546,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":",","depth":26,"bounds":{"left":0.16555852,"top":0.12529927,"width":0.002493351,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"bounds":{"left":0.17004654,"top":0.12689546,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", and","depth":26,"bounds":{"left":0.18700133,"top":0.12529927,"width":0.013131649,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"expire()","depth":27,"bounds":{"left":0.20212767,"top":0.12689546,"width":0.023936171,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"bounds":{"left":0.22805852,"top":0.12529927,"width":0.0013297872,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"bounds":{"left":0.12450133,"top":0.15403032,"width":0.040059842,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If multiple workers process the same","depth":26,"bounds":{"left":0.16456117,"top":0.15403032,"width":0.09275266,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$this->updateTarget","depth":27,"bounds":{"left":0.25930852,"top":0.15562649,"width":0.056848403,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"concurrently, they can both pass the","depth":26,"bounds":{"left":0.12450133,"top":0.15403032,"width":0.22855718,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"if (! is_null(Redis::get(...)))","depth":27,"bounds":{"left":0.18351063,"top":0.17478053,"width":0.09275266,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"check at the exact same millisecond. Both will then execute","depth":26,"bounds":{"left":0.12450133,"top":0.17318435,"width":0.21575798,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"bounds":{"left":0.2122673,"top":0.19393456,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and push duplicate metrics to Datadog.","depth":26,"bounds":{"left":0.22922207,"top":0.19233839,"width":0.09823803,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Fix:","depth":26,"bounds":{"left":0.12450133,"top":0.22106944,"width":0.019281914,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"This should be replaced with a single atomic operation using Redis's","depth":26,"bounds":{"left":0.14378324,"top":0.22106944,"width":0.16821809,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"NX","depth":27,"bounds":{"left":0.31399602,"top":0.22266561,"width":0.005984043,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Not eXists) and","depth":26,"bounds":{"left":0.12450133,"top":0.22106944,"width":0.22855718,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"EX","depth":27,"bounds":{"left":0.13696809,"top":0.24181964,"width":0.005984043,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Expire) flags, or Laravel’s built-in atomic cache locks:","depth":26,"bounds":{"left":0.14494681,"top":0.24022347,"width":0.13264628,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis::set($gaugeLockName, true, 'EX', 60, 'NX')","depth":27,"bounds":{"left":0.12649602,"top":0.26097366,"width":0.14361702,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"bounds":{"left":0.27210772,"top":0.25937748,"width":0.0013297872,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"3. Infinite Loops and Caller Contract","depth":23,"bounds":{"left":0.111369684,"top":0.30925778,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"3. Infinite Loops and Caller Contract","depth":24,"bounds":{"left":0.111369684,"top":0.30885875,"width":0.10571808,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Return Value Change:","depth":26,"bounds":{"left":0.12450133,"top":0.33599362,"width":0.055352394,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"bounds":{"left":0.18301196,"top":0.33758977,"width":0.06881649,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"previously returned whether it should iterate again based on staleness or chunk size. It now always returns","depth":26,"bounds":{"left":0.12450133,"top":0.33599362,"width":0.2237367,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"bounds":{"left":0.29438165,"top":0.3567438,"width":0.011968086,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(unless the selection is totally empty).","depth":26,"bounds":{"left":0.12450133,"top":0.35514766,"width":0.2130984,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"bounds":{"left":0.12450133,"top":0.40303272,"width":0.040059842,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the code calling this method relies on a","depth":26,"bounds":{"left":0.16456117,"top":0.40303272,"width":0.10305851,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"false","depth":27,"bounds":{"left":0.26961437,"top":0.4046289,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"return to pause, sleep, or gracefully terminate, changing this to an unconditional","depth":26,"bounds":{"left":0.12450133,"top":0.40303272,"width":0.22589761,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"bounds":{"left":0.26163563,"top":0.4237829,"width":0.011968086,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"could result in an aggressive \"busy loop\" that spikes CPU usage.","depth":26,"bounds":{"left":0.12450133,"top":0.42218676,"width":0.22290559,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"4. Performance Overhead","depth":23,"bounds":{"left":0.111369684,"top":0.49122107,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"4. Performance Overhead","depth":24,"bounds":{"left":0.111369684,"top":0.49082202,"width":0.076130316,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Expensive Garbage Collection:","depth":26,"bounds":{"left":0.12450133,"top":0.5179569,"width":0.078457445,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Calling","depth":26,"bounds":{"left":0.20295878,"top":0.5179569,"width":0.019281914,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles();","depth":27,"bounds":{"left":0.22423537,"top":0.51955307,"width":0.059840426,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"on","depth":26,"bounds":{"left":0.28607047,"top":0.5179569,"width":0.00880984,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"every single chunk","depth":26,"bounds":{"left":0.29488033,"top":0.5179569,"width":0.045545213,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.","depth":26,"bounds":{"left":0.12450133,"top":0.5179569,"width":0.23420878,"height":0.05546688},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Synchronous Metric Dispatch:","depth":26,"bounds":{"left":0.12450133,"top":0.584996,"width":0.07712766,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the","depth":26,"bounds":{"left":0.20162898,"top":0.584996,"width":0.015292553,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::increment","depth":27,"bounds":{"left":0.21891622,"top":0.5865922,"width":0.053856384,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"or","depth":26,"bounds":{"left":0.27476728,"top":0.584996,"width":0.0078125,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::gauge","depth":27,"bounds":{"left":0.28457448,"top":0.5865922,"width":0.041888297,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.","depth":26,"bounds":{"left":0.12450133,"top":0.584996,"width":0.23420878,"height":0.05546688},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"5. Lack of Explicit Error Handling","depth":23,"bounds":{"left":0.111369684,"top":0.67318434,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5. Lack of Explicit Error Handling","depth":24,"bounds":{"left":0.111369684,"top":0.67278534,"width":0.094913565,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"No","depth":26,"bounds":{"left":0.12450133,"top":0.6999202,"width":0.008477394,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"try/catch","depth":27,"bounds":{"left":0.1349734,"top":0.7015164,"width":0.026928192,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Blocks:","depth":26,"bounds":{"left":0.16389628,"top":0.6999202,"width":0.019614361,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The script executes complex network operations (","depth":26,"bounds":{"left":0.18351063,"top":0.6999202,"width":0.12367021,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"loadDocuments","depth":27,"bounds":{"left":0.30917552,"top":0.7015164,"width":0.038896278,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.","depth":26,"bounds":{"left":0.12450133,"top":0.6999202,"width":0.22888963,"height":0.09377494},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":22,"bounds":{"left":0.1087101,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":22,"bounds":{"left":0.124667555,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Redo","depth":22,"bounds":{"left":0.140625,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":21,"bounds":{"left":0.15658244,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":22,"bounds":{"left":0.17253989,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":21,"bounds":{"left":0.18849733,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXTextArea","text":"Ask Gemini","depth":21,"bounds":{"left":0.14594415,"top":0.86632085,"width":0.17021276,"height":0.01915403},"on_screen":true,"value":"Ask Gemini","help_text":"","role_description":"text entry area","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Ask Gemini","depth":22,"bounds":{"left":0.14594415,"top":0.8667199,"width":0.030418882,"height":0.018355945},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Upload & tools","depth":20,"bounds":{"left":0.12732713,"top":0.8567438,"width":0.015957447,"height":0.03830806},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Microphone","depth":20,"bounds":{"left":0.32413563,"top":0.86153233,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Send message","depth":20,"bounds":{"left":0.33211437,"top":0.8631285,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":false,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.","depth":17,"bounds":{"left":0.102726065,"top":0.9173983,"width":0.21775267,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Your privacy & Gemini Opens in a new window","depth":17,"bounds":{"left":0.32047874,"top":0.9173983,"width":0.043882977,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Your privacy & Gemini","depth":18,"bounds":{"left":0.32047874,"top":0.9173983,"width":0.043882977,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Opens in a new window","depth":19,"bounds":{"left":0.079288565,"top":0.91660017,"width":0.04720745,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Summarize page","depth":7,"bounds":{"left":0.08494016,"top":0.95730245,"width":0.053523935,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Summarize page","depth":9,"bounds":{"left":0.09059176,"top":0.96249,"width":0.042220745,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Skip to content","depth":6,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Skip to content","depth":7,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Open menu","depth":10,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"Homepage (g then d)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"jiminny","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"jiminny","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"app","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"app","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Search or jump to…","depth":9,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Type","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"/","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to search","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Chat with Copilot","depth":10,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXMenuButton","text":"Open Copilot…","depth":9,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXMenuButton","text":"Create new...","depth":9,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"All issues(g then i)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"All pull requests","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"All repositories","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"You have unread notifications(g then n)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Open user navigation menu","depth":9,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Repository navigation","depth":9,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Repository navigation","depth":10,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Code","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Code","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Pull requests (32)","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Pull requests","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"32","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Agents","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Agents","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Actions","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Actions","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Wiki","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Wiki","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Security and quality","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Security and quality","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Insights","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Insights","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Settings","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Settings","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"JY-20915 add alias for EU #12105 Edit title","depth":13,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"JY-20915 add alias for EU","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"#","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"12105","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Edit title","depth":14,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Awaiting approval","depth":13,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Awaiting approval","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXMenuButton","text":"Code","depth":13,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXStaticText","text":"Code","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Open","depth":13,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"LakyLak","depth":15,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"wants to merge 5 commits into","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"master","depth":15,"on_screen":false,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"master","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"from","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915-add-domain-specific-email-text-relay","depth":16,"on_screen":false,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915-add-domain-specific-email-text-relay","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy head branch name to clipboard","depth":16,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Lines changed: 365 additions & 1 deletion","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Conversation (1)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"Conversation","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"1","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Commits (5)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Commits","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Checks (3)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Checks","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"3","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Files changed (4)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Files changed","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"4","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Open","depth":14,"bounds":{"left":0.50332445,"top":0.07222666,"width":0.011968086,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"JY-20915 add alias for EU #12105 LakyLak wants to merge 5 commits into master from JY-20915-add-domain-specific-email-text-relay Copy head branch name to clipboard","depth":14,"bounds":{"left":0.5219415,"top":0.058260176,"width":0.22922207,"height":0.042298485},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915 add alias for EU","depth":16,"bounds":{"left":0.5219415,"top":0.05865922,"width":0.05817819,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915 add alias for EU","depth":17,"bounds":{"left":0.5219415,"top":0.06304868,"width":0.05817819,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"#","depth":16,"bounds":{"left":0.5827792,"top":0.06304868,"width":0.0028257978,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"12105","depth":16,"bounds":{"left":0.585605,"top":0.06304868,"width":0.012799202,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"LakyLak","depth":18,"bounds":{"left":0.5219415,"top":0.08339984,"width":0.016123671,"height":0.011971269},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":19,"bounds":{"left":0.5219415,"top":0.08339984,"width":0.016123671,"height":0.011971269},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"wants to merge 5 commits into","depth":18,"bounds":{"left":0.539395,"top":0.08339984,"width":0.058011968,"height":0.011971269},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"master","depth":18,"bounds":{"left":0.5987367,"top":0.08180367,"width":0.018450798,"height":0.015163607},"on_screen":true,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"master","depth":19,"bounds":{"left":0.6007314,"top":0.08339984,"width":0.014461436,"height":0.011572227},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"from","depth":19,"bounds":{"left":0.6185173,"top":0.08339984,"width":0.008643617,"height":0.011971269},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915-add-domain-specific-email-text-relay","depth":19,"bounds":{"left":0.6284907,"top":0.08180367,"width":0.112034574,"height":0.015163607},"on_screen":true,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915-add-domain-specific-email-text-relay","depth":20,"bounds":{"left":0.63048536,"top":0.08339984,"width":0.10804521,"height":0.011572227},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy head branch name to clipboard","depth":19,"bounds":{"left":0.741855,"top":0.07821229,"width":0.00930851,"height":0.022346368},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Conversation","depth":12,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Conversation","depth":13,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@LakyLak","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show options","depth":15,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXHeading","text":"LakyLak commented yesterday","depth":14,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXLink","text":"LakyLak","depth":16,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"commented","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"yesterday","depth":15,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"yesterday","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"JIRA: JY-20915","depth":16,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"JIRA:","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915","depth":17,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Changes:","depth":16,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Changes:","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Support text relay for EU","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Prevent errors from EU on US","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Add or remove reactions","depth":16,"on_screen":false,"help_text":"","role_description":"summary","subrole":"AXSummary","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"LakyLak","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"added","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"commits","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"yesterday","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"yesterday","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@LakyLak","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"JY-20915","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"add alias for EU","depth":14,"on_screen":false,"help_text":"JY-20915 add alias for EU","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"add alias for EU","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"8 / 10 checks OK","depth":14,"on_screen":false,"help_text":"","role_description":"summary","subrole":"AXSummary","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"e5255d5","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"e5255d5","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@LakyLak","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"JY-20915","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"fix env recognization","depth":14,"on_screen":false,"help_text":"JY-20915 fix env recognization","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"fix env recognization","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"11 / 12 checks OK","depth":14,"on_screen":false,"help_text":"","role_description":"summary","subrole":"AXSummary","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"8953054","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"8953054","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@LakyLak","depth":12,"bounds":{"left":0.5249335,"top":0.0,"width":0.0066489363,"height":0.015961692},"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"JY-20915","depth":14,"bounds":{"left":0.53357714,"top":0.0,"width":0.019115692,"height":0.011572227},"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915","depth":15,"bounds":{"left":0.53357714,"top":0.0,"width":0.019115692,"height":0.011572227},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"code smells & tests","depth":14,"bounds":{"left":0.55502,"top":0.0,"width":0.045711435,"height":0.011572227},"on_screen":false,"help_text":"JY-20915 code smells & tests","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"code smells & tests","depth":15,"bounds":{"left":0.55502,"top":0.0,"width":0.045711435,"height":0.011572227},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"11 / 12 checks OK","depth":14,"bounds":{"left":0.7563165,"top":0.0,"width":0.005319149,"height":0.016759777},"on_screen":false,"help_text":"","role_description":"summary","subrole":"AXSummary","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"73d4bea","depth":14,"bounds":{"left":0.76296544,"top":0.0,"width":0.016954787,"height":0.011572227},"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"73d4bea","depth":15,"bounds":{"left":0.76296544,"top":0.0,"width":0.016954787,"height":0.011572227},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@LakyLak","depth":12,"bounds":{"left":0.5249335,"top":0.0,"width":0.0066489363,"height":0.015961692},"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"JY-20915","depth":14,"bounds":{"left":0.53357714,"top":0.0,"width":0.019115692,"height":0.011572227},"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915","depth":15,"bounds":{"left":0.53357714,"top":0.0,"width":0.019115692,"height":0.011572227},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"add more tests","depth":14,"bounds":{"left":0.55502,"top":0.0,"width":0.03357713,"height":0.011572227},"on_screen":false,"help_text":"JY-20915 add more tests","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"add more tests","depth":15,"bounds":{"left":0.55502,"top":0.0,"width":0.03357713,"height":0.011572227},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"11 / 12 checks OK","depth":14,"bounds":{"left":0.7563165,"top":0.0,"width":0.005319149,"height":0.016759777},"on_screen":false,"help_text":"","role_description":"summary","subrole":"AXSummary","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"5d2dd3e","depth":14,"bounds":{"left":0.76296544,"top":0.0,"width":0.016954787,"height":0.011572227},"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"5d2dd3e","depth":15,"bounds":{"left":0.76296544,"top":0.0,"width":0.016954787,"height":0.011572227},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@LakyLak","depth":12,"bounds":{"left":0.5249335,"top":0.0,"width":0.0066489363,"height":0.015961692},"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"JY-20915","depth":14,"bounds":{"left":0.53357714,"top":0.0,"width":0.019115692,"height":0.011572227},"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915","depth":15,"bounds":{"left":0.53357714,"top":0.0,"width":0.019115692,"height":0.011572227},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"add sms received test","depth":14,"bounds":{"left":0.55502,"top":0.0,"width":0.050531916,"height":0.011572227},"on_screen":false,"help_text":"JY-20915 add sms received test","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"add sms received test","depth":15,"bounds":{"left":0.55502,"top":0.0,"width":0.050531916,"height":0.011572227},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"12 / 12 checks OK","depth":14,"bounds":{"left":0.7563165,"top":0.0,"width":0.005319149,"height":0.016759777},"on_screen":false,"help_text":"","role_description":"summary","subrole":"AXSummary","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"b8c8eb5","depth":14,"bounds":{"left":0.76296544,"top":0.0,"width":0.016954787,"height":0.011572227},"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"b8c8eb5","depth":15,"bounds":{"left":0.76296544,"top":0.0,"width":0.016954787,"height":0.011572227},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@sonarqubecloud","depth":13,"bounds":{"left":0.49268618,"top":0.019952115,"width":0.013297873,"height":0.031923383},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show options","depth":14,"bounds":{"left":0.7662899,"top":0.0207502,"width":0.007978723,"height":0.02952913},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXHeading","text":"sonarqubecloud Bot commented 3 hours ago","depth":13,"bounds":{"left":0.5169548,"top":0.0207502,"width":0.24135639,"height":0.029928172},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXLink","text":"sonarqubecloud","depth":15,"bounds":{"left":0.5169548,"top":0.028731046,"width":0.036236703,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"sonarqubecloud","depth":16,"bounds":{"left":0.5169548,"top":0.028731046,"width":0.036236703,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"}]...
|
-6028027651937307624
|
-5678133564466170088
|
idle
|
accessibility
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo
Share & export
Copy
Show more options
Ask Gemini
Ask Gemini
Upload & tools
Microphone
Send message
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Summarize page
Summarize page
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
All issues(g then i)
All pull requests
All repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Pull requests (32)
Pull requests
(
32
)
Agents
Agents
Actions
Actions
Wiki
Wiki
Security and quality
Security and quality
Insights
Insights
Settings
Settings
JY-20915 add alias for EU #12105 Edit title
JY-20915 add alias for EU
#
12105
Edit title
Awaiting approval
Awaiting approval
Code
Code
Open
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Lines changed: 365 additions & 1 deletion
Conversation (1)
Conversation
(
1
)
Commits (5)
Commits
(
5
)
Checks (3)
Checks
(
3
)
Files changed (4)
Files changed
(
4
)
Open
JY-20915 add alias for EU #12105 LakyLak wants to merge 5 commits into master from JY-20915-add-domain-specific-email-text-relay Copy head branch name to clipboard
JY-20915 add alias for EU
JY-20915 add alias for EU
#
12105
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Conversation
Conversation
@LakyLak
Show options
LakyLak commented yesterday
LakyLak
LakyLak
commented
yesterday
yesterday
JIRA: JY-20915
JIRA:
JY-20915
JY-20915
Changes:
Changes:
Support text relay for EU
Prevent errors from EU on US
Add or remove reactions
LakyLak
LakyLak
added
5
commits
yesterday
yesterday
@LakyLak
JY-20915
JY-20915
add alias for EU
add alias for EU
8 / 10 checks OK
e5255d5
e5255d5
@LakyLak
JY-20915
JY-20915
fix env recognization
fix env recognization
11 / 12 checks OK
8953054
8953054
@LakyLak
JY-20915
JY-20915
code smells & tests
code smells & tests
11 / 12 checks OK
73d4bea
73d4bea
@LakyLak
JY-20915
JY-20915
add more tests
add more tests
11 / 12 checks OK
5d2dd3e
5d2dd3e
@LakyLak
JY-20915
JY-20915
add sms received test
add sms received test
12 / 12 checks OK
b8c8eb5
b8c8eb5
@sonarqubecloud
Show options
sonarqubecloud Bot commented 3 hours ago
sonarqubecloud
sonarqubecloud...
|
NULL
|
NULL
|
NULL
|
NULL
|
|
65239
|
2287
|
3
|
2026-05-20T15:31:47.899758+00:00
|
/Users/lukas/.screenpipe/data/data/2026-05-20/1779 /Users/lukas/.screenpipe/data/data/2026-05-20/1779291107899_m1.jpg...
|
Firefox
|
JY-20915 add alias for EU by LakyLak · Pull Reques JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app — Work...
|
1
|
github.com/jiminny/app/pull/12105
|
monitor_1
|
NULL
|
NULL
|
NULL
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo
Share & export
Copy
Show more options
Ask Gemini
Ask Gemini
Upload & tools
Microphone
Send message
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Summarize page
Summarize page
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
All issues(g then i)
All pull requests
All repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Pull requests (32)
Pull requests
(
32
)
Agents
Agents
Actions
Actions
Wiki
Wiki
Security and quality
Security and quality
Insights
Insights
Settings
Settings
JY-20915 add alias for EU #12105 Edit title
JY-20915 add alias for EU
#
12105
Edit title
Awaiting approval
Awaiting approval
Code
Code
Open
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Lines changed: 365 additions & 1 deletion
Conversation (1)
Conversation
(
1
)
Commits (5)
Commits
(
5
)
Checks (3)
Checks
(
3
)
Files changed (4)
Files changed
(
4
)
Open
JY-20915 add alias for EU #12105 LakyLak wants to merge 5 commits into master from JY-20915-add-domain-specific-email-text-relay Copy head branch name to clipboard
JY-20915 add alias for EU
JY-20915 add alias for EU
#
12105
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Conversation
Conversation
@LakyLak
Show options
LakyLak commented yesterday
LakyLak
LakyLak
commented
yesterday
yesterday
JIRA: JY-20915
JIRA:
JY-20915
JY-20915
Changes:
Changes:
Support text relay for EU
Prevent errors from EU on US
Add or remove reactions
LakyLak
LakyLak
added
5
commits
yesterday
yesterday
@LakyLak
JY-20915
JY-20915
add alias for EU
add alias for EU
8 / 10 checks OK
e5255d5
e5255d5
@LakyLak
JY-20915
JY-20915
fix env recognization
fix env recognization
11 / 12 checks OK
8953054
8953054
@LakyLak
JY-20915...
|
[{"role":"AXRadioButton","text [{"role":"AXRadioButton","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Close tab","depth":5,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXRadioButton","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Pipelines - jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Pipelines - jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Tab","depth":4,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Customize sidebar","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Close Google Gemini (⌃X)","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Tabs from other devices","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open history (⇧⌘H)","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open bookmarks (⌘B)","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"AI Chat settings","depth":7,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Close","depth":7,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Main menu","depth":13,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"New chat","depth":12,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Gemini","depth":15,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Chat","depth":13,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Open menu for conversation actions.","depth":13,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Conversation with Gemini","depth":15,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Conversation with Gemini","depth":16,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"You said I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.","depth":23,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Expand","depth":21,"on_screen":false,"role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a concise summary of the changes made in the code selection:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Refactoring and Dependencies","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Refactoring and Dependencies","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Replaced Interface with Implementation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Changed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusInterface","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusManager","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"in the imports and constructor dependency injection.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Redis Support:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Imported","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Illuminate\\Support\\Facades\\Redis","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to handle a newly introduced memory-logging lock mechanism.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Traits:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"WorkerIdTrait","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", along with the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"worker","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"field tracking in the final logs.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Loop and Worker Lifecycle Adjustments","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Loop and Worker Lifecycle Adjustments","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Worker Auto-Restart:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Deleted the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property, the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"constant, and the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"shouldIterateAgain()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method. The script no longer tracks execution iterations or triggers a self-managed worker restart.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Simplified Chunk Method Return:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Modified","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to always return","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Priority Handling for Throttled Entities","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Priority Handling for Throttled Entities","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Stale Record Elevation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Updated the logic in","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"rescheduleThrottledEntities()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to check if a throttled entity group is classified as \"stale\". If it is, the entities are rescheduled using","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"QueuePriorityEnum::NORMAL","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"; otherwise, they keep their respective queue priority.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Metrics, Logging, and Environment Handling","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Metrics, Logging, and Environment Handling","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Environment Tracking:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Introduced a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$environment","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property (defaulting to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"'local'","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") and a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"setEnvironment()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method to dynamically assign it.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Memory Monitoring via Datadog Gauge:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method that submits peak memory allocation to Datadog as a gauge metric (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"jiminny.reindex-memory-usage","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis Lock for Metrics Throttling:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Implemented a Redis-backed lock (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"%s-process-manager-lock","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") with a 60-second expiration inside","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to prevent over-reporting metrics.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Log Payload Update:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added total processed items (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$updatedDocuments + $deletedDocuments","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") to the final information log.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":23,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"You said are there any issues or potential bottlenecks","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"are there any issues or potential bottlenecks","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Edit","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a breakdown of the potential issues to look out for:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"1. Memory Leak Risk (OOM Crashes)","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"1. Memory Leak Risk (OOM Crashes)","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Restart Logic:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The most significant change is the removal of the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Dependency Leaks:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If any underlying service (like the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ElasticSearchClient","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", ORM, or Datadog helper) retains references to processed entities,","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Hardcoded Limits:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The code still includes","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ini_set('memory_limit', '512M');","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":". Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"2. Race Condition in Redis Lock","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"2. Race Condition in Redis Lock","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Non-Atomic Operations:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method uses three distinct steps to handle the lock:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"get()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":",","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", and","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"expire()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If multiple workers process the same","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$this->updateTarget","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"concurrently, they can both pass the","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"if (! is_null(Redis::get(...)))","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"check at the exact same millisecond. Both will then execute","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and push duplicate metrics to Datadog.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Fix:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"This should be replaced with a single atomic operation using Redis's","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"NX","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Not eXists) and","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"EX","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Expire) flags, or Laravel’s built-in atomic cache locks:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis::set($gaugeLockName, true, 'EX', 60, 'NX')","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"3. Infinite Loops and Caller Contract","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"3. Infinite Loops and Caller Contract","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Return Value Change:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"previously returned whether it should iterate again based on staleness or chunk size. It now always returns","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(unless the selection is totally empty).","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the code calling this method relies on a","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"false","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"return to pause, sleep, or gracefully terminate, changing this to an unconditional","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"could result in an aggressive \"busy loop\" that spikes CPU usage.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"4. Performance Overhead","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"4. Performance Overhead","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Expensive Garbage Collection:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Calling","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles();","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"on","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"every single chunk","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Synchronous Metric Dispatch:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::increment","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"or","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::gauge","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"5. Lack of Explicit Error Handling","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5. Lack of Explicit Error Handling","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"No","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"try/catch","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Blocks:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The script executes complex network operations (","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"loadDocuments","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":22,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":22,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Redo","depth":22,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":21,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":22,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":21,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXTextArea","text":"Ask Gemini","depth":21,"on_screen":true,"value":"Ask Gemini","help_text":"","role_description":"text entry area","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Ask Gemini","depth":22,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Upload & tools","depth":20,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Microphone","depth":20,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Send message","depth":20,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":false,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.","depth":17,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Your privacy & Gemini Opens in a new window","depth":17,"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Your privacy & Gemini","depth":18,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Opens in a new window","depth":19,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Summarize page","depth":7,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Summarize page","depth":9,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Skip to content","depth":6,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Skip to content","depth":7,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Open menu","depth":10,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"Homepage (g then d)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"jiminny","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"jiminny","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"app","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"app","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Search or jump to…","depth":9,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Type","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"/","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to search","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Chat with Copilot","depth":10,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXMenuButton","text":"Open Copilot…","depth":9,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXMenuButton","text":"Create new...","depth":9,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"All issues(g then i)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"All pull requests","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"All repositories","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"You have unread notifications(g then n)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Open user navigation menu","depth":9,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Repository navigation","depth":9,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Repository navigation","depth":10,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Code","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Code","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Pull requests (32)","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Pull requests","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"32","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Agents","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Agents","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Actions","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Actions","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Wiki","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Wiki","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Security and quality","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Security and quality","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Insights","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Insights","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Settings","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Settings","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"JY-20915 add alias for EU #12105 Edit title","depth":13,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"JY-20915 add alias for EU","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"#","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"12105","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Edit title","depth":14,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Awaiting approval","depth":13,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Awaiting approval","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXMenuButton","text":"Code","depth":13,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXStaticText","text":"Code","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Open","depth":13,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"LakyLak","depth":15,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"wants to merge 5 commits into","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"master","depth":15,"on_screen":false,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"master","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"from","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915-add-domain-specific-email-text-relay","depth":16,"on_screen":false,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915-add-domain-specific-email-text-relay","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy head branch name to clipboard","depth":16,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Lines changed: 365 additions & 1 deletion","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Conversation (1)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"Conversation","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"1","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Commits (5)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Commits","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Checks (3)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Checks","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"3","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Files changed (4)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Files changed","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"4","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Open","depth":14,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"JY-20915 add alias for EU #12105 LakyLak wants to merge 5 commits into master from JY-20915-add-domain-specific-email-text-relay Copy head branch name to clipboard","depth":14,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915 add alias for EU","depth":16,"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915 add alias for EU","depth":17,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"#","depth":16,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"12105","depth":16,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"LakyLak","depth":18,"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":19,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"wants to merge 5 commits into","depth":18,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"master","depth":18,"on_screen":true,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"master","depth":19,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"from","depth":19,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915-add-domain-specific-email-text-relay","depth":19,"on_screen":true,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915-add-domain-specific-email-text-relay","depth":20,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy head branch name to clipboard","depth":19,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Conversation","depth":12,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Conversation","depth":13,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@LakyLak","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show options","depth":15,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXHeading","text":"LakyLak commented yesterday","depth":14,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXLink","text":"LakyLak","depth":16,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"commented","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"yesterday","depth":15,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"yesterday","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"JIRA: JY-20915","depth":16,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"JIRA:","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915","depth":17,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Changes:","depth":16,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Changes:","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Support text relay for EU","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Prevent errors from EU on US","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Add or remove reactions","depth":16,"on_screen":false,"help_text":"","role_description":"summary","subrole":"AXSummary","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"LakyLak","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"added","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"commits","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"yesterday","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"yesterday","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@LakyLak","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"JY-20915","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"add alias for EU","depth":14,"on_screen":false,"help_text":"JY-20915 add alias for EU","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"add alias for EU","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"8 / 10 checks OK","depth":14,"on_screen":false,"help_text":"","role_description":"summary","subrole":"AXSummary","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"e5255d5","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"e5255d5","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@LakyLak","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"JY-20915","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"fix env recognization","depth":14,"on_screen":false,"help_text":"JY-20915 fix env recognization","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"fix env recognization","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"11 / 12 checks OK","depth":14,"on_screen":false,"help_text":"","role_description":"summary","subrole":"AXSummary","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"8953054","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"8953054","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@LakyLak","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"JY-20915","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false}]...
|
-7201230878671898169
|
-1048292959795775720
|
idle
|
accessibility
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo
Share & export
Copy
Show more options
Ask Gemini
Ask Gemini
Upload & tools
Microphone
Send message
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Summarize page
Summarize page
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
All issues(g then i)
All pull requests
All repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Pull requests (32)
Pull requests
(
32
)
Agents
Agents
Actions
Actions
Wiki
Wiki
Security and quality
Security and quality
Insights
Insights
Settings
Settings
JY-20915 add alias for EU #12105 Edit title
JY-20915 add alias for EU
#
12105
Edit title
Awaiting approval
Awaiting approval
Code
Code
Open
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Lines changed: 365 additions & 1 deletion
Conversation (1)
Conversation
(
1
)
Commits (5)
Commits
(
5
)
Checks (3)
Checks
(
3
)
Files changed (4)
Files changed
(
4
)
Open
JY-20915 add alias for EU #12105 LakyLak wants to merge 5 commits into master from JY-20915-add-domain-specific-email-text-relay Copy head branch name to clipboard
JY-20915 add alias for EU
JY-20915 add alias for EU
#
12105
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Conversation
Conversation
@LakyLak
Show options
LakyLak commented yesterday
LakyLak
LakyLak
commented
yesterday
yesterday
JIRA: JY-20915
JIRA:
JY-20915
JY-20915
Changes:
Changes:
Support text relay for EU
Prevent errors from EU on US
Add or remove reactions
LakyLak
LakyLak
added
5
commits
yesterday
yesterday
@LakyLak
JY-20915
JY-20915
add alias for EU
add alias for EU
8 / 10 checks OK
e5255d5
e5255d5
@LakyLak
JY-20915
JY-20915
fix env recognization
fix env recognization
11 / 12 checks OK
8953054
8953054
@LakyLak
JY-20915...
|
NULL
|
NULL
|
NULL
|
NULL
|
|
65240
|
2288
|
3
|
2026-05-20T15:31:59.468635+00:00
|
/Users/lukas/.screenpipe/data/data/2026-05-20/1779 /Users/lukas/.screenpipe/data/data/2026-05-20/1779291119468_m2.jpg...
|
Firefox
|
JY-20915 add alias for EU by LakyLak · Pull Reques JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app — Work...
|
1
|
github.com/jiminny/app/pull/12105
|
monitor_2
|
NULL
|
NULL
|
NULL
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo
Share & export
Copy
Show more options
Ask Gemini
Ask Gemini
Upload & tools
Microphone
Send message
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Summarize page
Summarize page
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
All issues(g then i)
All pull requests
All repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Pull requests (32)
Pull requests
(
32
)
Agents
Agents
Actions
Actions
Wiki
Wiki
Security and quality
Security and quality
Insights
Insights
Settings
Settings
JY-20915 add alias for EU #12105 Edit title
JY-20915 add alias for EU
#
12105
Edit title
Awaiting approval
Awaiting approval
Code
Code
Open
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Lines changed: 365 additions & 1 deletion
Conversation (1)
Conversation
(
1
)
Commits (5)
Commits
(
5
)
Checks (3)
Checks
(
3
)
Files changed (4)
Files changed
(
4
)
Open
JY-20915 add alias for EU #12105 LakyLak wants to merge 5 commits into master from JY-20915-add-domain-specific-email-text-relay Copy head branch name to clipboard
JY-20915 add alias for EU
JY-20915 add alias for EU
#
12105
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Conversation
Conversation
@LakyLak
Show options
LakyLak commented yesterday
LakyLak
LakyLak
commented
yesterday
yesterday
JIRA: JY-20915
JIRA:
JY-20915
JY-20915
Changes:
Changes:
Support text relay for EU
Prevent errors from EU on US
Add or remove reactions
LakyLak
LakyLak
added
5
commits
yesterday...
|
[{"role":"AXRadioButton","text [{"role":"AXRadioButton","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":4,"bounds":{"left":0.0,"top":0.0518755,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.06304868,"width":0.10106383,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":4,"bounds":{"left":0.0,"top":0.08459697,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.09577015,"width":0.10721409,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.11731844,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.12849163,"width":0.15791224,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.15003991,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.16121309,"width":0.1796875,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.18276137,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.19393456,"width":0.12699468,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Close tab","depth":5,"bounds":{"left":0.06732048,"top":0.18994413,"width":0.007978723,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXRadioButton","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":4,"bounds":{"left":0.0,"top":0.21548285,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":5,"bounds":{"left":0.013297873,"top":0.22665602,"width":0.10139628,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":4,"bounds":{"left":0.0,"top":0.2482043,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.25937748,"width":0.18068483,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":4,"bounds":{"left":0.0,"top":0.28092578,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":5,"bounds":{"left":0.013297873,"top":0.29209897,"width":0.4644282,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Pipelines - jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.31364724,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Pipelines - jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.32482043,"width":0.039228722,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.3463687,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.3575419,"width":0.15824468,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Tab","depth":4,"bounds":{"left":0.0028257978,"top":0.38068634,"width":0.07413564,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Customize sidebar","depth":6,"bounds":{"left":0.0028257978,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Close Google Gemini (⌃X)","depth":6,"bounds":{"left":0.013796543,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Tabs from other devices","depth":6,"bounds":{"left":0.024933511,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open history (⇧⌘H)","depth":6,"bounds":{"left":0.036070477,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open bookmarks (⌘B)","depth":6,"bounds":{"left":0.04720745,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"AI Chat settings","depth":7,"bounds":{"left":0.35854387,"top":0.055067837,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Close","depth":7,"bounds":{"left":0.37051198,"top":0.055067837,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Main menu","depth":13,"bounds":{"left":0.08228058,"top":0.09736632,"width":0.015957447,"height":0.03830806},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"New chat","depth":12,"bounds":{"left":0.09823803,"top":0.10215483,"width":0.028091755,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Gemini","depth":15,"bounds":{"left":0.1008976,"top":0.10654429,"width":0.02144282,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Chat","depth":13,"bounds":{"left":0.35854387,"top":0.10215483,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Open menu for conversation actions.","depth":13,"bounds":{"left":0.37051198,"top":0.10215483,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Conversation with Gemini","depth":15,"bounds":{"left":0.079288565,"top":0.14445332,"width":0.0003324468,"height":0.0007980846},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Conversation with Gemini","depth":16,"bounds":{"left":0.079288565,"top":0.14684756,"width":0.1200133,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"You said I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.","depth":23,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Expand","depth":21,"on_screen":false,"role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a concise summary of the changes made in the code selection:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Refactoring and Dependencies","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Refactoring and Dependencies","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Replaced Interface with Implementation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Changed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusInterface","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusManager","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"in the imports and constructor dependency injection.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Redis Support:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Imported","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Illuminate\\Support\\Facades\\Redis","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to handle a newly introduced memory-logging lock mechanism.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Traits:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"WorkerIdTrait","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", along with the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"worker","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"field tracking in the final logs.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Loop and Worker Lifecycle Adjustments","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Loop and Worker Lifecycle Adjustments","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Worker Auto-Restart:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Deleted the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property, the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"constant, and the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"shouldIterateAgain()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method. The script no longer tracks execution iterations or triggers a self-managed worker restart.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Simplified Chunk Method Return:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Modified","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to always return","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Priority Handling for Throttled Entities","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Priority Handling for Throttled Entities","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Stale Record Elevation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Updated the logic in","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"rescheduleThrottledEntities()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to check if a throttled entity group is classified as \"stale\". If it is, the entities are rescheduled using","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"QueuePriorityEnum::NORMAL","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"; otherwise, they keep their respective queue priority.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Metrics, Logging, and Environment Handling","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Metrics, Logging, and Environment Handling","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Environment Tracking:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Introduced a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$environment","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property (defaulting to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"'local'","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") and a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"setEnvironment()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method to dynamically assign it.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Memory Monitoring via Datadog Gauge:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method that submits peak memory allocation to Datadog as a gauge metric (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"jiminny.reindex-memory-usage","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis Lock for Metrics Throttling:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Implemented a Redis-backed lock (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"%s-process-manager-lock","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") with a 60-second expiration inside","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to prevent over-reporting metrics.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Log Payload Update:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added total processed items (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$updatedDocuments + $deletedDocuments","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") to the final information log.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":23,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"You said are there any issues or potential bottlenecks","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"are there any issues or potential bottlenecks","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Edit","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a breakdown of the potential issues to look out for:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"1. Memory Leak Risk (OOM Crashes)","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"1. Memory Leak Risk (OOM Crashes)","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Restart Logic:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The most significant change is the removal of the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Dependency Leaks:","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.050033245,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If any underlying service (like the","depth":26,"bounds":{"left":0.17453457,"top":0.0,"width":0.08277926,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ElasticSearchClient","depth":27,"bounds":{"left":0.25930852,"top":0.0,"width":0.056848403,"height":0.015163607},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", ORM, or Datadog helper) retains references to processed entities,","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.21775267,"height":0.03631285},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"bounds":{"left":0.26811835,"top":0.0,"width":0.056848403,"height":0.015163607},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.23138298,"height":0.05546688},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Hardcoded Limits:","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.047041222,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The code still includes","depth":26,"bounds":{"left":0.17154256,"top":0.0,"width":0.05651596,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ini_set('memory_limit', '512M');","depth":27,"bounds":{"left":0.23005319,"top":0.0,"width":0.095744684,"height":0.015163607},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":". Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.23055187,"height":0.05546688},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"2. Race Condition in Redis Lock","depth":23,"bounds":{"left":0.111369684,"top":0.07940942,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"2. Race Condition in Redis Lock","depth":24,"bounds":{"left":0.111369684,"top":0.079010375,"width":0.09125665,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Non-Atomic Operations:","depth":26,"bounds":{"left":0.12450133,"top":0.10614525,"width":0.062333778,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The","depth":26,"bounds":{"left":0.18683511,"top":0.10614525,"width":0.011635638,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage","depth":27,"bounds":{"left":0.20046543,"top":0.10774142,"width":0.041888297,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method uses three distinct steps to handle the lock:","depth":26,"bounds":{"left":0.12450133,"top":0.10614525,"width":0.22722739,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"get()","depth":27,"bounds":{"left":0.14860372,"top":0.12689546,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":",","depth":26,"bounds":{"left":0.16555852,"top":0.12529927,"width":0.002493351,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"bounds":{"left":0.17004654,"top":0.12689546,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", and","depth":26,"bounds":{"left":0.18700133,"top":0.12529927,"width":0.013131649,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"expire()","depth":27,"bounds":{"left":0.20212767,"top":0.12689546,"width":0.023936171,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"bounds":{"left":0.22805852,"top":0.12529927,"width":0.0013297872,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"bounds":{"left":0.12450133,"top":0.15403032,"width":0.040059842,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If multiple workers process the same","depth":26,"bounds":{"left":0.16456117,"top":0.15403032,"width":0.09275266,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$this->updateTarget","depth":27,"bounds":{"left":0.25930852,"top":0.15562649,"width":0.056848403,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"concurrently, they can both pass the","depth":26,"bounds":{"left":0.12450133,"top":0.15403032,"width":0.22855718,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"if (! is_null(Redis::get(...)))","depth":27,"bounds":{"left":0.18351063,"top":0.17478053,"width":0.09275266,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"check at the exact same millisecond. Both will then execute","depth":26,"bounds":{"left":0.12450133,"top":0.17318435,"width":0.21575798,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"bounds":{"left":0.2122673,"top":0.19393456,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and push duplicate metrics to Datadog.","depth":26,"bounds":{"left":0.22922207,"top":0.19233839,"width":0.09823803,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Fix:","depth":26,"bounds":{"left":0.12450133,"top":0.22106944,"width":0.019281914,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"This should be replaced with a single atomic operation using Redis's","depth":26,"bounds":{"left":0.14378324,"top":0.22106944,"width":0.16821809,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"NX","depth":27,"bounds":{"left":0.31399602,"top":0.22266561,"width":0.005984043,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Not eXists) and","depth":26,"bounds":{"left":0.12450133,"top":0.22106944,"width":0.22855718,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"EX","depth":27,"bounds":{"left":0.13696809,"top":0.24181964,"width":0.005984043,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Expire) flags, or Laravel’s built-in atomic cache locks:","depth":26,"bounds":{"left":0.14494681,"top":0.24022347,"width":0.13264628,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis::set($gaugeLockName, true, 'EX', 60, 'NX')","depth":27,"bounds":{"left":0.12649602,"top":0.26097366,"width":0.14361702,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"bounds":{"left":0.27210772,"top":0.25937748,"width":0.0013297872,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"3. Infinite Loops and Caller Contract","depth":23,"bounds":{"left":0.111369684,"top":0.30925778,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"3. Infinite Loops and Caller Contract","depth":24,"bounds":{"left":0.111369684,"top":0.30885875,"width":0.10571808,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Return Value Change:","depth":26,"bounds":{"left":0.12450133,"top":0.33599362,"width":0.055352394,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"bounds":{"left":0.18301196,"top":0.33758977,"width":0.06881649,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"previously returned whether it should iterate again based on staleness or chunk size. It now always returns","depth":26,"bounds":{"left":0.12450133,"top":0.33599362,"width":0.2237367,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"bounds":{"left":0.29438165,"top":0.3567438,"width":0.011968086,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(unless the selection is totally empty).","depth":26,"bounds":{"left":0.12450133,"top":0.35514766,"width":0.2130984,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"bounds":{"left":0.12450133,"top":0.40303272,"width":0.040059842,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the code calling this method relies on a","depth":26,"bounds":{"left":0.16456117,"top":0.40303272,"width":0.10305851,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"false","depth":27,"bounds":{"left":0.26961437,"top":0.4046289,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"return to pause, sleep, or gracefully terminate, changing this to an unconditional","depth":26,"bounds":{"left":0.12450133,"top":0.40303272,"width":0.22589761,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"bounds":{"left":0.26163563,"top":0.4237829,"width":0.011968086,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"could result in an aggressive \"busy loop\" that spikes CPU usage.","depth":26,"bounds":{"left":0.12450133,"top":0.42218676,"width":0.22290559,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"4. Performance Overhead","depth":23,"bounds":{"left":0.111369684,"top":0.49122107,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"4. Performance Overhead","depth":24,"bounds":{"left":0.111369684,"top":0.49082202,"width":0.076130316,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Expensive Garbage Collection:","depth":26,"bounds":{"left":0.12450133,"top":0.5179569,"width":0.078457445,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Calling","depth":26,"bounds":{"left":0.20295878,"top":0.5179569,"width":0.019281914,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles();","depth":27,"bounds":{"left":0.22423537,"top":0.51955307,"width":0.059840426,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"on","depth":26,"bounds":{"left":0.28607047,"top":0.5179569,"width":0.00880984,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"every single chunk","depth":26,"bounds":{"left":0.29488033,"top":0.5179569,"width":0.045545213,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.","depth":26,"bounds":{"left":0.12450133,"top":0.5179569,"width":0.23420878,"height":0.05546688},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Synchronous Metric Dispatch:","depth":26,"bounds":{"left":0.12450133,"top":0.584996,"width":0.07712766,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the","depth":26,"bounds":{"left":0.20162898,"top":0.584996,"width":0.015292553,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::increment","depth":27,"bounds":{"left":0.21891622,"top":0.5865922,"width":0.053856384,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"or","depth":26,"bounds":{"left":0.27476728,"top":0.584996,"width":0.0078125,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::gauge","depth":27,"bounds":{"left":0.28457448,"top":0.5865922,"width":0.041888297,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.","depth":26,"bounds":{"left":0.12450133,"top":0.584996,"width":0.23420878,"height":0.05546688},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"5. Lack of Explicit Error Handling","depth":23,"bounds":{"left":0.111369684,"top":0.67318434,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5. Lack of Explicit Error Handling","depth":24,"bounds":{"left":0.111369684,"top":0.67278534,"width":0.094913565,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"No","depth":26,"bounds":{"left":0.12450133,"top":0.6999202,"width":0.008477394,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"try/catch","depth":27,"bounds":{"left":0.1349734,"top":0.7015164,"width":0.026928192,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Blocks:","depth":26,"bounds":{"left":0.16389628,"top":0.6999202,"width":0.019614361,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The script executes complex network operations (","depth":26,"bounds":{"left":0.18351063,"top":0.6999202,"width":0.12367021,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"loadDocuments","depth":27,"bounds":{"left":0.30917552,"top":0.7015164,"width":0.038896278,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.","depth":26,"bounds":{"left":0.12450133,"top":0.6999202,"width":0.22888963,"height":0.09377494},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":22,"bounds":{"left":0.1087101,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":22,"bounds":{"left":0.124667555,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Redo","depth":22,"bounds":{"left":0.140625,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":21,"bounds":{"left":0.15658244,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":22,"bounds":{"left":0.17253989,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":21,"bounds":{"left":0.18849733,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXTextArea","text":"Ask Gemini","depth":21,"bounds":{"left":0.14594415,"top":0.86632085,"width":0.17021276,"height":0.01915403},"on_screen":true,"value":"Ask Gemini","help_text":"","role_description":"text entry area","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Ask Gemini","depth":22,"bounds":{"left":0.14594415,"top":0.8667199,"width":0.030418882,"height":0.018355945},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Upload & tools","depth":20,"bounds":{"left":0.12732713,"top":0.8567438,"width":0.015957447,"height":0.03830806},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Microphone","depth":20,"bounds":{"left":0.32413563,"top":0.86153233,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Send message","depth":20,"bounds":{"left":0.33211437,"top":0.8631285,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":false,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.","depth":17,"bounds":{"left":0.102726065,"top":0.9173983,"width":0.21775267,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Your privacy & Gemini Opens in a new window","depth":17,"bounds":{"left":0.32047874,"top":0.9173983,"width":0.043882977,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Your privacy & Gemini","depth":18,"bounds":{"left":0.32047874,"top":0.9173983,"width":0.043882977,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Opens in a new window","depth":19,"bounds":{"left":0.079288565,"top":0.91660017,"width":0.04720745,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Summarize page","depth":7,"bounds":{"left":0.08494016,"top":0.95730245,"width":0.053523935,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Summarize page","depth":9,"bounds":{"left":0.09059176,"top":0.96249,"width":0.042220745,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Skip to content","depth":6,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Skip to content","depth":7,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Open menu","depth":10,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"Homepage (g then d)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"jiminny","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"jiminny","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"app","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"app","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Search or jump to…","depth":9,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Type","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"/","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to search","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Chat with Copilot","depth":10,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXMenuButton","text":"Open Copilot…","depth":9,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXMenuButton","text":"Create new...","depth":9,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"All issues(g then i)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"All pull requests","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"All repositories","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"You have unread notifications(g then n)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Open user navigation menu","depth":9,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Repository navigation","depth":9,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Repository navigation","depth":10,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Code","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Code","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Pull requests (32)","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Pull requests","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"32","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Agents","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Agents","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Actions","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Actions","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Wiki","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Wiki","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Security and quality","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Security and quality","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Insights","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Insights","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Settings","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Settings","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"JY-20915 add alias for EU #12105 Edit title","depth":13,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"JY-20915 add alias for EU","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"#","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"12105","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Edit title","depth":14,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Awaiting approval","depth":13,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Awaiting approval","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXMenuButton","text":"Code","depth":13,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXStaticText","text":"Code","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Open","depth":13,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"LakyLak","depth":15,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"wants to merge 5 commits into","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"master","depth":15,"on_screen":false,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"master","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"from","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915-add-domain-specific-email-text-relay","depth":16,"on_screen":false,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915-add-domain-specific-email-text-relay","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy head branch name to clipboard","depth":16,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Lines changed: 365 additions & 1 deletion","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Conversation (1)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"Conversation","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"1","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Commits (5)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Commits","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Checks (3)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Checks","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"3","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Files changed (4)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Files changed","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"4","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Open","depth":14,"bounds":{"left":0.50332445,"top":0.07222666,"width":0.011968086,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"JY-20915 add alias for EU #12105 LakyLak wants to merge 5 commits into master from JY-20915-add-domain-specific-email-text-relay Copy head branch name to clipboard","depth":14,"bounds":{"left":0.5219415,"top":0.058260176,"width":0.22922207,"height":0.042298485},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915 add alias for EU","depth":16,"bounds":{"left":0.5219415,"top":0.05865922,"width":0.05817819,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915 add alias for EU","depth":17,"bounds":{"left":0.5219415,"top":0.06304868,"width":0.05817819,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"#","depth":16,"bounds":{"left":0.5827792,"top":0.06304868,"width":0.0028257978,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"12105","depth":16,"bounds":{"left":0.585605,"top":0.06304868,"width":0.012799202,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"LakyLak","depth":18,"bounds":{"left":0.5219415,"top":0.08339984,"width":0.016123671,"height":0.011971269},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":19,"bounds":{"left":0.5219415,"top":0.08339984,"width":0.016123671,"height":0.011971269},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"wants to merge 5 commits into","depth":18,"bounds":{"left":0.539395,"top":0.08339984,"width":0.058011968,"height":0.011971269},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"master","depth":18,"bounds":{"left":0.5987367,"top":0.08180367,"width":0.018450798,"height":0.015163607},"on_screen":true,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"master","depth":19,"bounds":{"left":0.6007314,"top":0.08339984,"width":0.014461436,"height":0.011572227},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"from","depth":19,"bounds":{"left":0.6185173,"top":0.08339984,"width":0.008643617,"height":0.011971269},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915-add-domain-specific-email-text-relay","depth":19,"bounds":{"left":0.6284907,"top":0.08180367,"width":0.112034574,"height":0.015163607},"on_screen":true,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915-add-domain-specific-email-text-relay","depth":20,"bounds":{"left":0.63048536,"top":0.08339984,"width":0.10804521,"height":0.011572227},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy head branch name to clipboard","depth":19,"bounds":{"left":0.741855,"top":0.07821229,"width":0.00930851,"height":0.022346368},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Conversation","depth":12,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Conversation","depth":13,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@LakyLak","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show options","depth":15,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXHeading","text":"LakyLak commented yesterday","depth":14,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXLink","text":"LakyLak","depth":16,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"commented","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"yesterday","depth":15,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"yesterday","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"JIRA: JY-20915","depth":16,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"JIRA:","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915","depth":17,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Changes:","depth":16,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Changes:","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Support text relay for EU","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Prevent errors from EU on US","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Add or remove reactions","depth":16,"on_screen":false,"help_text":"","role_description":"summary","subrole":"AXSummary","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"LakyLak","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"added","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"commits","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"yesterday","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false}]...
|
-5917428508800079207
|
-1048292410041026792
|
idle
|
accessibility
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo
Share & export
Copy
Show more options
Ask Gemini
Ask Gemini
Upload & tools
Microphone
Send message
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Summarize page
Summarize page
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
All issues(g then i)
All pull requests
All repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Pull requests (32)
Pull requests
(
32
)
Agents
Agents
Actions
Actions
Wiki
Wiki
Security and quality
Security and quality
Insights
Insights
Settings
Settings
JY-20915 add alias for EU #12105 Edit title
JY-20915 add alias for EU
#
12105
Edit title
Awaiting approval
Awaiting approval
Code
Code
Open
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Lines changed: 365 additions & 1 deletion
Conversation (1)
Conversation
(
1
)
Commits (5)
Commits
(
5
)
Checks (3)
Checks
(
3
)
Files changed (4)
Files changed
(
4
)
Open
JY-20915 add alias for EU #12105 LakyLak wants to merge 5 commits into master from JY-20915-add-domain-specific-email-text-relay Copy head branch name to clipboard
JY-20915 add alias for EU
JY-20915 add alias for EU
#
12105
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Conversation
Conversation
@LakyLak
Show options
LakyLak commented yesterday
LakyLak
LakyLak
commented
yesterday
yesterday
JIRA: JY-20915
JIRA:
JY-20915
JY-20915
Changes:
Changes:
Support text relay for EU
Prevent errors from EU on US
Add or remove reactions
LakyLak
LakyLak
added
5
commits
yesterday...
|
65238
|
NULL
|
NULL
|
NULL
|
|
65241
|
2287
|
4
|
2026-05-20T15:32:18.257600+00:00
|
/Users/lukas/.screenpipe/data/data/2026-05-20/1779 /Users/lukas/.screenpipe/data/data/2026-05-20/1779291138257_m1.jpg...
|
Firefox
|
JY-20915 add alias for EU by LakyLak · Pull Reques JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app — Work...
|
1
|
github.com/jiminny/app/pull/12105
|
monitor_1
|
NULL
|
NULL
|
NULL
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo
Share & export
Copy
Show more options
Ask Gemini
Ask Gemini
Upload & tools
Microphone
Send message
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Summarize page
Summarize page
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
All issues(g then i)
All pull requests
All repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Pull requests (32)
Pull requests
(
32
)
Agents
Agents
Actions
Actions
Wiki
Wiki
Security and quality
Security and quality
Insights
Insights
Settings
Settings
JY-20915 add alias for EU #12105 Edit title
JY-20915 add alias for EU
#
12105
Edit title
Awaiting approval
Awaiting approval
Code
Code
Open
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Lines changed: 365 additions & 1 deletion
Conversation (1)
Conversation
(
1
)
Commits (5)
Commits
(
5
)
Checks (3)
Checks
(
3
)
Files changed (4)
Files changed
(
4
)
Open
JY-20915 add alias for EU #12105 LakyLak wants to merge 5 commits into master from JY-20915-add-domain-specific-email-text-relay Copy head branch name to clipboard
JY-20915 add alias for EU
JY-20915 add alias for EU
#
12105
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay...
|
[{"role":"AXRadioButton","text [{"role":"AXRadioButton","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Close tab","depth":5,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXRadioButton","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Pipelines - jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Pipelines - jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Tab","depth":4,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Customize sidebar","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Close Google Gemini (⌃X)","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Tabs from other devices","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open history (⇧⌘H)","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open bookmarks (⌘B)","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"AI Chat settings","depth":7,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Close","depth":7,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Main menu","depth":13,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"New chat","depth":12,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Gemini","depth":15,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Chat","depth":13,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Open menu for conversation actions.","depth":13,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Conversation with Gemini","depth":15,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Conversation with Gemini","depth":16,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"You said I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.","depth":23,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Expand","depth":21,"on_screen":false,"role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a concise summary of the changes made in the code selection:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Refactoring and Dependencies","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Refactoring and Dependencies","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Replaced Interface with Implementation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Changed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusInterface","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusManager","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"in the imports and constructor dependency injection.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Redis Support:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Imported","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Illuminate\\Support\\Facades\\Redis","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to handle a newly introduced memory-logging lock mechanism.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Traits:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"WorkerIdTrait","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", along with the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"worker","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"field tracking in the final logs.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Loop and Worker Lifecycle Adjustments","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Loop and Worker Lifecycle Adjustments","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Worker Auto-Restart:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Deleted the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property, the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"constant, and the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"shouldIterateAgain()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method. The script no longer tracks execution iterations or triggers a self-managed worker restart.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Simplified Chunk Method Return:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Modified","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to always return","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Priority Handling for Throttled Entities","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Priority Handling for Throttled Entities","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Stale Record Elevation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Updated the logic in","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"rescheduleThrottledEntities()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to check if a throttled entity group is classified as \"stale\". If it is, the entities are rescheduled using","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"QueuePriorityEnum::NORMAL","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"; otherwise, they keep their respective queue priority.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Metrics, Logging, and Environment Handling","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Metrics, Logging, and Environment Handling","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Environment Tracking:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Introduced a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$environment","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property (defaulting to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"'local'","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") and a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"setEnvironment()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method to dynamically assign it.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Memory Monitoring via Datadog Gauge:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method that submits peak memory allocation to Datadog as a gauge metric (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"jiminny.reindex-memory-usage","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis Lock for Metrics Throttling:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Implemented a Redis-backed lock (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"%s-process-manager-lock","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") with a 60-second expiration inside","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to prevent over-reporting metrics.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Log Payload Update:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added total processed items (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$updatedDocuments + $deletedDocuments","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") to the final information log.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":23,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"You said are there any issues or potential bottlenecks","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"are there any issues or potential bottlenecks","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Edit","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a breakdown of the potential issues to look out for:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"1. Memory Leak Risk (OOM Crashes)","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"1. Memory Leak Risk (OOM Crashes)","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Restart Logic:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The most significant change is the removal of the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Dependency Leaks:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If any underlying service (like the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ElasticSearchClient","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", ORM, or Datadog helper) retains references to processed entities,","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Hardcoded Limits:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The code still includes","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ini_set('memory_limit', '512M');","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":". Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"2. Race Condition in Redis Lock","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"2. Race Condition in Redis Lock","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Non-Atomic Operations:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method uses three distinct steps to handle the lock:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"get()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":",","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", and","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"expire()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If multiple workers process the same","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$this->updateTarget","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"concurrently, they can both pass the","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"if (! is_null(Redis::get(...)))","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"check at the exact same millisecond. Both will then execute","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and push duplicate metrics to Datadog.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Fix:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"This should be replaced with a single atomic operation using Redis's","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"NX","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Not eXists) and","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"EX","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Expire) flags, or Laravel’s built-in atomic cache locks:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis::set($gaugeLockName, true, 'EX', 60, 'NX')","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"3. Infinite Loops and Caller Contract","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"3. Infinite Loops and Caller Contract","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Return Value Change:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"previously returned whether it should iterate again based on staleness or chunk size. It now always returns","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(unless the selection is totally empty).","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the code calling this method relies on a","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"false","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"return to pause, sleep, or gracefully terminate, changing this to an unconditional","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"could result in an aggressive \"busy loop\" that spikes CPU usage.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"4. Performance Overhead","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"4. Performance Overhead","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Expensive Garbage Collection:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Calling","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles();","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"on","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"every single chunk","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Synchronous Metric Dispatch:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::increment","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"or","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::gauge","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"5. Lack of Explicit Error Handling","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5. Lack of Explicit Error Handling","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"No","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"try/catch","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Blocks:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The script executes complex network operations (","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"loadDocuments","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":22,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":22,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Redo","depth":22,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":21,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":22,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":21,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXTextArea","text":"Ask Gemini","depth":21,"on_screen":true,"value":"Ask Gemini","help_text":"","role_description":"text entry area","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Ask Gemini","depth":22,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Upload & tools","depth":20,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Microphone","depth":20,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Send message","depth":20,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":false,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.","depth":17,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Your privacy & Gemini Opens in a new window","depth":17,"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Your privacy & Gemini","depth":18,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Opens in a new window","depth":19,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Summarize page","depth":7,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Summarize page","depth":9,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Skip to content","depth":6,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Skip to content","depth":7,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Open menu","depth":10,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"Homepage (g then d)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"jiminny","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"jiminny","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"app","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"app","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Search or jump to…","depth":9,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Type","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"/","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to search","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Chat with Copilot","depth":10,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXMenuButton","text":"Open Copilot…","depth":9,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXMenuButton","text":"Create new...","depth":9,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"All issues(g then i)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"All pull requests","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"All repositories","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"You have unread notifications(g then n)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Open user navigation menu","depth":9,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Repository navigation","depth":9,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Repository navigation","depth":10,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Code","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Code","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Pull requests (32)","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Pull requests","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"32","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Agents","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Agents","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Actions","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Actions","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Wiki","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Wiki","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Security and quality","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Security and quality","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Insights","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Insights","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Settings","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Settings","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"JY-20915 add alias for EU #12105 Edit title","depth":13,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"JY-20915 add alias for EU","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"#","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"12105","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Edit title","depth":14,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Awaiting approval","depth":13,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Awaiting approval","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXMenuButton","text":"Code","depth":13,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXStaticText","text":"Code","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Open","depth":13,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"LakyLak","depth":15,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"wants to merge 5 commits into","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"master","depth":15,"on_screen":false,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"master","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"from","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915-add-domain-specific-email-text-relay","depth":16,"on_screen":false,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915-add-domain-specific-email-text-relay","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy head branch name to clipboard","depth":16,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Lines changed: 365 additions & 1 deletion","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Conversation (1)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"Conversation","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"1","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Commits (5)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Commits","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Checks (3)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Checks","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"3","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Files changed (4)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Files changed","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"4","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Open","depth":14,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"JY-20915 add alias for EU #12105 LakyLak wants to merge 5 commits into master from JY-20915-add-domain-specific-email-text-relay Copy head branch name to clipboard","depth":14,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915 add alias for EU","depth":16,"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915 add alias for EU","depth":17,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"#","depth":16,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"12105","depth":16,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"LakyLak","depth":18,"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":19,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"wants to merge 5 commits into","depth":18,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"master","depth":18,"on_screen":true,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"master","depth":19,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"from","depth":19,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915-add-domain-specific-email-text-relay","depth":19,"on_screen":true,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915-add-domain-specific-email-text-relay","depth":20,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"}]...
|
2104575741699509463
|
-1048292410044172516
|
idle
|
accessibility
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo
Share & export
Copy
Show more options
Ask Gemini
Ask Gemini
Upload & tools
Microphone
Send message
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Summarize page
Summarize page
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
All issues(g then i)
All pull requests
All repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Pull requests (32)
Pull requests
(
32
)
Agents
Agents
Actions
Actions
Wiki
Wiki
Security and quality
Security and quality
Insights
Insights
Settings
Settings
JY-20915 add alias for EU #12105 Edit title
JY-20915 add alias for EU
#
12105
Edit title
Awaiting approval
Awaiting approval
Code
Code
Open
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Lines changed: 365 additions & 1 deletion
Conversation (1)
Conversation
(
1
)
Commits (5)
Commits
(
5
)
Checks (3)
Checks
(
3
)
Files changed (4)
Files changed
(
4
)
Open
JY-20915 add alias for EU #12105 LakyLak wants to merge 5 commits into master from JY-20915-add-domain-specific-email-text-relay Copy head branch name to clipboard
JY-20915 add alias for EU
JY-20915 add alias for EU
#
12105
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay...
|
65239
|
NULL
|
NULL
|
NULL
|
|
65242
|
2288
|
4
|
2026-05-20T15:32:29.951418+00:00
|
/Users/lukas/.screenpipe/data/data/2026-05-20/1779 /Users/lukas/.screenpipe/data/data/2026-05-20/1779291149951_m2.jpg...
|
Firefox
|
JY-20915 add alias for EU by LakyLak · Pull Reques JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app — Work...
|
1
|
github.com/jiminny/app/pull/12105
|
monitor_2
|
NULL
|
NULL
|
NULL
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo
Share & export
Copy
Show more options
Ask Gemini
Ask Gemini
Upload & tools
Microphone
Send message
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Summarize page
Summarize page
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
All issues(g then i)
All pull requests
All repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Pull requests (32)
Pull requests
(
32
)
Agents
Agents
Actions
Actions
Wiki
Wiki
Security and quality
Security and quality
Insights
Insights
Settings
Settings
JY-20915 add alias for EU #12105 Edit title
JY-20915 add alias for EU
#
12105
Edit title
Awaiting approval
Awaiting approval
Code
Code
Open
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Lines changed: 365 additions & 1 deletion
Conversation (1)
Conversation
(
1
)
Commits (5)
Commits
(
5
)
Checks (3)
Checks
(
3
)
Files changed (4)
Files changed
(
4
)
Open
JY-20915 add alias for EU #12105 LakyLak wants to merge 5 commits into master from JY-20915-add-domain-specific-email-text-relay Copy head branch name to clipboard
JY-20915 add alias for EU
JY-20915 add alias for EU
#
12105
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Conversation
Conversation
@LakyLak
Show options
LakyLak commented yesterday
LakyLak
LakyLak
commented
yesterday
yesterday
JIRA: JY-20915
JIRA:
JY-20915
JY-20915
Changes:
Changes:
Support text relay for EU
Prevent errors from EU on US
Add or remove reactions
LakyLak
LakyLak
added
5
commits
yesterday
yesterday
@LakyLak
JY-20915
JY-20915
add alias for EU
add alias for EU
8 / 10 checks OK
e5255d5
e5255d5
@LakyLak
JY-20915
JY-20915
fix env recognization
fix env recognization
11 / 12 checks OK
8953054
8953054
@LakyLak
JY-20915
JY-20915
code smells & tests
code smells & tests
11 / 12 checks OK
73d4bea
73d4bea
@LakyLak
JY-20915
JY-20915
add more tests
add more tests
11 / 12 checks OK
5d2dd3e
5d2dd3e
@LakyLak
JY-20915
JY-20915
add sms received test
add sms received test
12 / 12 checks OK
b8c8eb5
b8c8eb5
@sonarqubecloud
Show options
sonarqubecloud Bot commented 3 hours ago
sonarqubecloud
sonarqubecloud
Bot
commented
3 hours ago...
|
[{"role":"AXRadioButton","text [{"role":"AXRadioButton","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":4,"bounds":{"left":0.0,"top":0.0518755,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.06304868,"width":0.10106383,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":4,"bounds":{"left":0.0,"top":0.08459697,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.09577015,"width":0.10721409,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.11731844,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.12849163,"width":0.15791224,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.15003991,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.16121309,"width":0.1796875,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.18276137,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.19393456,"width":0.12699468,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Close tab","depth":5,"bounds":{"left":0.06732048,"top":0.18994413,"width":0.007978723,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXRadioButton","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":4,"bounds":{"left":0.0,"top":0.21548285,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":5,"bounds":{"left":0.013297873,"top":0.22665602,"width":0.10139628,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":4,"bounds":{"left":0.0,"top":0.2482043,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.25937748,"width":0.18068483,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":4,"bounds":{"left":0.0,"top":0.28092578,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":5,"bounds":{"left":0.013297873,"top":0.29209897,"width":0.4644282,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Pipelines - jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.31364724,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Pipelines - jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.32482043,"width":0.039228722,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.3463687,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.3575419,"width":0.15824468,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Tab","depth":4,"bounds":{"left":0.0028257978,"top":0.38068634,"width":0.07413564,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Customize sidebar","depth":6,"bounds":{"left":0.0028257978,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Close Google Gemini (⌃X)","depth":6,"bounds":{"left":0.013796543,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Tabs from other devices","depth":6,"bounds":{"left":0.024933511,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open history (⇧⌘H)","depth":6,"bounds":{"left":0.036070477,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open bookmarks (⌘B)","depth":6,"bounds":{"left":0.04720745,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"AI Chat settings","depth":7,"bounds":{"left":0.35854387,"top":0.055067837,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Close","depth":7,"bounds":{"left":0.37051198,"top":0.055067837,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Main menu","depth":13,"bounds":{"left":0.08228058,"top":0.09736632,"width":0.015957447,"height":0.03830806},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"New chat","depth":12,"bounds":{"left":0.09823803,"top":0.10215483,"width":0.028091755,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Gemini","depth":15,"bounds":{"left":0.1008976,"top":0.10654429,"width":0.02144282,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Chat","depth":13,"bounds":{"left":0.35854387,"top":0.10215483,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Open menu for conversation actions.","depth":13,"bounds":{"left":0.37051198,"top":0.10215483,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Conversation with Gemini","depth":15,"bounds":{"left":0.079288565,"top":0.14445332,"width":0.0003324468,"height":0.0007980846},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Conversation with Gemini","depth":16,"bounds":{"left":0.079288565,"top":0.14684756,"width":0.1200133,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"You said I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.","depth":23,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Expand","depth":21,"on_screen":false,"role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a concise summary of the changes made in the code selection:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Refactoring and Dependencies","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Refactoring and Dependencies","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Replaced Interface with Implementation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Changed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusInterface","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusManager","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"in the imports and constructor dependency injection.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Redis Support:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Imported","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Illuminate\\Support\\Facades\\Redis","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to handle a newly introduced memory-logging lock mechanism.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Traits:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"WorkerIdTrait","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", along with the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"worker","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"field tracking in the final logs.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Loop and Worker Lifecycle Adjustments","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Loop and Worker Lifecycle Adjustments","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Worker Auto-Restart:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Deleted the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property, the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"constant, and the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"shouldIterateAgain()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method. The script no longer tracks execution iterations or triggers a self-managed worker restart.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Simplified Chunk Method Return:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Modified","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to always return","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Priority Handling for Throttled Entities","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Priority Handling for Throttled Entities","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Stale Record Elevation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Updated the logic in","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"rescheduleThrottledEntities()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to check if a throttled entity group is classified as \"stale\". If it is, the entities are rescheduled using","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"QueuePriorityEnum::NORMAL","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"; otherwise, they keep their respective queue priority.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Metrics, Logging, and Environment Handling","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Metrics, Logging, and Environment Handling","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Environment Tracking:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Introduced a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$environment","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property (defaulting to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"'local'","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") and a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"setEnvironment()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method to dynamically assign it.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Memory Monitoring via Datadog Gauge:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method that submits peak memory allocation to Datadog as a gauge metric (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"jiminny.reindex-memory-usage","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis Lock for Metrics Throttling:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Implemented a Redis-backed lock (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"%s-process-manager-lock","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") with a 60-second expiration inside","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to prevent over-reporting metrics.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Log Payload Update:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added total processed items (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$updatedDocuments + $deletedDocuments","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") to the final information log.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":23,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"You said are there any issues or potential bottlenecks","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"are there any issues or potential bottlenecks","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Edit","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a breakdown of the potential issues to look out for:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"1. Memory Leak Risk (OOM Crashes)","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"1. Memory Leak Risk (OOM Crashes)","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Restart Logic:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The most significant change is the removal of the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Dependency Leaks:","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.050033245,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If any underlying service (like the","depth":26,"bounds":{"left":0.17453457,"top":0.0,"width":0.08277926,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ElasticSearchClient","depth":27,"bounds":{"left":0.25930852,"top":0.0,"width":0.056848403,"height":0.015163607},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", ORM, or Datadog helper) retains references to processed entities,","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.21775267,"height":0.03631285},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"bounds":{"left":0.26811835,"top":0.0,"width":0.056848403,"height":0.015163607},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.23138298,"height":0.05546688},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Hardcoded Limits:","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.047041222,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The code still includes","depth":26,"bounds":{"left":0.17154256,"top":0.0,"width":0.05651596,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ini_set('memory_limit', '512M');","depth":27,"bounds":{"left":0.23005319,"top":0.0,"width":0.095744684,"height":0.015163607},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":". Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.23055187,"height":0.05546688},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"2. Race Condition in Redis Lock","depth":23,"bounds":{"left":0.111369684,"top":0.07940942,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"2. Race Condition in Redis Lock","depth":24,"bounds":{"left":0.111369684,"top":0.079010375,"width":0.09125665,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Non-Atomic Operations:","depth":26,"bounds":{"left":0.12450133,"top":0.10614525,"width":0.062333778,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The","depth":26,"bounds":{"left":0.18683511,"top":0.10614525,"width":0.011635638,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage","depth":27,"bounds":{"left":0.20046543,"top":0.10774142,"width":0.041888297,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method uses three distinct steps to handle the lock:","depth":26,"bounds":{"left":0.12450133,"top":0.10614525,"width":0.22722739,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"get()","depth":27,"bounds":{"left":0.14860372,"top":0.12689546,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":",","depth":26,"bounds":{"left":0.16555852,"top":0.12529927,"width":0.002493351,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"bounds":{"left":0.17004654,"top":0.12689546,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", and","depth":26,"bounds":{"left":0.18700133,"top":0.12529927,"width":0.013131649,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"expire()","depth":27,"bounds":{"left":0.20212767,"top":0.12689546,"width":0.023936171,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"bounds":{"left":0.22805852,"top":0.12529927,"width":0.0013297872,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"bounds":{"left":0.12450133,"top":0.15403032,"width":0.040059842,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If multiple workers process the same","depth":26,"bounds":{"left":0.16456117,"top":0.15403032,"width":0.09275266,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$this->updateTarget","depth":27,"bounds":{"left":0.25930852,"top":0.15562649,"width":0.056848403,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"concurrently, they can both pass the","depth":26,"bounds":{"left":0.12450133,"top":0.15403032,"width":0.22855718,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"if (! is_null(Redis::get(...)))","depth":27,"bounds":{"left":0.18351063,"top":0.17478053,"width":0.09275266,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"check at the exact same millisecond. Both will then execute","depth":26,"bounds":{"left":0.12450133,"top":0.17318435,"width":0.21575798,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"bounds":{"left":0.2122673,"top":0.19393456,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and push duplicate metrics to Datadog.","depth":26,"bounds":{"left":0.22922207,"top":0.19233839,"width":0.09823803,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Fix:","depth":26,"bounds":{"left":0.12450133,"top":0.22106944,"width":0.019281914,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"This should be replaced with a single atomic operation using Redis's","depth":26,"bounds":{"left":0.14378324,"top":0.22106944,"width":0.16821809,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"NX","depth":27,"bounds":{"left":0.31399602,"top":0.22266561,"width":0.005984043,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Not eXists) and","depth":26,"bounds":{"left":0.12450133,"top":0.22106944,"width":0.22855718,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"EX","depth":27,"bounds":{"left":0.13696809,"top":0.24181964,"width":0.005984043,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Expire) flags, or Laravel’s built-in atomic cache locks:","depth":26,"bounds":{"left":0.14494681,"top":0.24022347,"width":0.13264628,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis::set($gaugeLockName, true, 'EX', 60, 'NX')","depth":27,"bounds":{"left":0.12649602,"top":0.26097366,"width":0.14361702,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"bounds":{"left":0.27210772,"top":0.25937748,"width":0.0013297872,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"3. Infinite Loops and Caller Contract","depth":23,"bounds":{"left":0.111369684,"top":0.30925778,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"3. Infinite Loops and Caller Contract","depth":24,"bounds":{"left":0.111369684,"top":0.30885875,"width":0.10571808,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Return Value Change:","depth":26,"bounds":{"left":0.12450133,"top":0.33599362,"width":0.055352394,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"bounds":{"left":0.18301196,"top":0.33758977,"width":0.06881649,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"previously returned whether it should iterate again based on staleness or chunk size. It now always returns","depth":26,"bounds":{"left":0.12450133,"top":0.33599362,"width":0.2237367,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"bounds":{"left":0.29438165,"top":0.3567438,"width":0.011968086,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(unless the selection is totally empty).","depth":26,"bounds":{"left":0.12450133,"top":0.35514766,"width":0.2130984,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"bounds":{"left":0.12450133,"top":0.40303272,"width":0.040059842,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the code calling this method relies on a","depth":26,"bounds":{"left":0.16456117,"top":0.40303272,"width":0.10305851,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"false","depth":27,"bounds":{"left":0.26961437,"top":0.4046289,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"return to pause, sleep, or gracefully terminate, changing this to an unconditional","depth":26,"bounds":{"left":0.12450133,"top":0.40303272,"width":0.22589761,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"bounds":{"left":0.26163563,"top":0.4237829,"width":0.011968086,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"could result in an aggressive \"busy loop\" that spikes CPU usage.","depth":26,"bounds":{"left":0.12450133,"top":0.42218676,"width":0.22290559,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"4. Performance Overhead","depth":23,"bounds":{"left":0.111369684,"top":0.49122107,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"4. Performance Overhead","depth":24,"bounds":{"left":0.111369684,"top":0.49082202,"width":0.076130316,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Expensive Garbage Collection:","depth":26,"bounds":{"left":0.12450133,"top":0.5179569,"width":0.078457445,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Calling","depth":26,"bounds":{"left":0.20295878,"top":0.5179569,"width":0.019281914,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles();","depth":27,"bounds":{"left":0.22423537,"top":0.51955307,"width":0.059840426,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"on","depth":26,"bounds":{"left":0.28607047,"top":0.5179569,"width":0.00880984,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"every single chunk","depth":26,"bounds":{"left":0.29488033,"top":0.5179569,"width":0.045545213,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.","depth":26,"bounds":{"left":0.12450133,"top":0.5179569,"width":0.23420878,"height":0.05546688},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Synchronous Metric Dispatch:","depth":26,"bounds":{"left":0.12450133,"top":0.584996,"width":0.07712766,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the","depth":26,"bounds":{"left":0.20162898,"top":0.584996,"width":0.015292553,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::increment","depth":27,"bounds":{"left":0.21891622,"top":0.5865922,"width":0.053856384,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"or","depth":26,"bounds":{"left":0.27476728,"top":0.584996,"width":0.0078125,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::gauge","depth":27,"bounds":{"left":0.28457448,"top":0.5865922,"width":0.041888297,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.","depth":26,"bounds":{"left":0.12450133,"top":0.584996,"width":0.23420878,"height":0.05546688},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"5. Lack of Explicit Error Handling","depth":23,"bounds":{"left":0.111369684,"top":0.67318434,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5. Lack of Explicit Error Handling","depth":24,"bounds":{"left":0.111369684,"top":0.67278534,"width":0.094913565,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"No","depth":26,"bounds":{"left":0.12450133,"top":0.6999202,"width":0.008477394,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"try/catch","depth":27,"bounds":{"left":0.1349734,"top":0.7015164,"width":0.026928192,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Blocks:","depth":26,"bounds":{"left":0.16389628,"top":0.6999202,"width":0.019614361,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The script executes complex network operations (","depth":26,"bounds":{"left":0.18351063,"top":0.6999202,"width":0.12367021,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"loadDocuments","depth":27,"bounds":{"left":0.30917552,"top":0.7015164,"width":0.038896278,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.","depth":26,"bounds":{"left":0.12450133,"top":0.6999202,"width":0.22888963,"height":0.09377494},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":22,"bounds":{"left":0.1087101,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":22,"bounds":{"left":0.124667555,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Redo","depth":22,"bounds":{"left":0.140625,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":21,"bounds":{"left":0.15658244,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":22,"bounds":{"left":0.17253989,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":21,"bounds":{"left":0.18849733,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXTextArea","text":"Ask Gemini","depth":21,"bounds":{"left":0.14594415,"top":0.86632085,"width":0.17021276,"height":0.01915403},"on_screen":true,"value":"Ask Gemini","help_text":"","role_description":"text entry area","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Ask Gemini","depth":22,"bounds":{"left":0.14594415,"top":0.8667199,"width":0.030418882,"height":0.018355945},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Upload & tools","depth":20,"bounds":{"left":0.12732713,"top":0.8567438,"width":0.015957447,"height":0.03830806},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Microphone","depth":20,"bounds":{"left":0.32413563,"top":0.86153233,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Send message","depth":20,"bounds":{"left":0.33211437,"top":0.8631285,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":false,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.","depth":17,"bounds":{"left":0.102726065,"top":0.9173983,"width":0.21775267,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Your privacy & Gemini Opens in a new window","depth":17,"bounds":{"left":0.32047874,"top":0.9173983,"width":0.043882977,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Your privacy & Gemini","depth":18,"bounds":{"left":0.32047874,"top":0.9173983,"width":0.043882977,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Opens in a new window","depth":19,"bounds":{"left":0.079288565,"top":0.91660017,"width":0.04720745,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Summarize page","depth":7,"bounds":{"left":0.08494016,"top":0.95730245,"width":0.053523935,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Summarize page","depth":9,"bounds":{"left":0.09059176,"top":0.96249,"width":0.042220745,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Skip to content","depth":6,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Skip to content","depth":7,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Open menu","depth":10,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"Homepage (g then d)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"jiminny","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"jiminny","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"app","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"app","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Search or jump to…","depth":9,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Type","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"/","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to search","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Chat with Copilot","depth":10,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXMenuButton","text":"Open Copilot…","depth":9,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXMenuButton","text":"Create new...","depth":9,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"All issues(g then i)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"All pull requests","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"All repositories","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"You have unread notifications(g then n)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Open user navigation menu","depth":9,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Repository navigation","depth":9,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Repository navigation","depth":10,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Code","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Code","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Pull requests (32)","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Pull requests","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"32","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Agents","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Agents","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Actions","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Actions","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Wiki","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Wiki","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Security and quality","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Security and quality","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Insights","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Insights","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Settings","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Settings","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"JY-20915 add alias for EU #12105 Edit title","depth":13,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"JY-20915 add alias for EU","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"#","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"12105","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Edit title","depth":14,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Awaiting approval","depth":13,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Awaiting approval","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXMenuButton","text":"Code","depth":13,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXStaticText","text":"Code","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Open","depth":13,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"LakyLak","depth":15,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"wants to merge 5 commits into","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"master","depth":15,"on_screen":false,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"master","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"from","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915-add-domain-specific-email-text-relay","depth":16,"on_screen":false,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915-add-domain-specific-email-text-relay","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy head branch name to clipboard","depth":16,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Lines changed: 365 additions & 1 deletion","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Conversation (1)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"Conversation","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"1","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Commits (5)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Commits","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Checks (3)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Checks","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"3","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Files changed (4)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Files changed","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"4","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Open","depth":14,"bounds":{"left":0.50332445,"top":0.07222666,"width":0.011968086,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"JY-20915 add alias for EU #12105 LakyLak wants to merge 5 commits into master from JY-20915-add-domain-specific-email-text-relay Copy head branch name to clipboard","depth":14,"bounds":{"left":0.5219415,"top":0.058260176,"width":0.22922207,"height":0.042298485},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915 add alias for EU","depth":16,"bounds":{"left":0.5219415,"top":0.05865922,"width":0.05817819,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915 add alias for EU","depth":17,"bounds":{"left":0.5219415,"top":0.06304868,"width":0.05817819,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"#","depth":16,"bounds":{"left":0.5827792,"top":0.06304868,"width":0.0028257978,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"12105","depth":16,"bounds":{"left":0.585605,"top":0.06304868,"width":0.012799202,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"LakyLak","depth":18,"bounds":{"left":0.5219415,"top":0.08339984,"width":0.016123671,"height":0.011971269},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":19,"bounds":{"left":0.5219415,"top":0.08339984,"width":0.016123671,"height":0.011971269},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"wants to merge 5 commits into","depth":18,"bounds":{"left":0.539395,"top":0.08339984,"width":0.058011968,"height":0.011971269},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"master","depth":18,"bounds":{"left":0.5987367,"top":0.08180367,"width":0.018450798,"height":0.015163607},"on_screen":true,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"master","depth":19,"bounds":{"left":0.6007314,"top":0.08339984,"width":0.014461436,"height":0.011572227},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"from","depth":19,"bounds":{"left":0.6185173,"top":0.08339984,"width":0.008643617,"height":0.011971269},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915-add-domain-specific-email-text-relay","depth":19,"bounds":{"left":0.6284907,"top":0.08180367,"width":0.112034574,"height":0.015163607},"on_screen":true,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915-add-domain-specific-email-text-relay","depth":20,"bounds":{"left":0.63048536,"top":0.08339984,"width":0.10804521,"height":0.011572227},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy head branch name to clipboard","depth":19,"bounds":{"left":0.741855,"top":0.07821229,"width":0.00930851,"height":0.022346368},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Conversation","depth":12,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Conversation","depth":13,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@LakyLak","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show options","depth":15,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXHeading","text":"LakyLak commented yesterday","depth":14,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXLink","text":"LakyLak","depth":16,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"commented","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"yesterday","depth":15,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"yesterday","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"JIRA: JY-20915","depth":16,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"JIRA:","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915","depth":17,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Changes:","depth":16,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Changes:","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Support text relay for EU","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Prevent errors from EU on US","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Add or remove reactions","depth":16,"on_screen":false,"help_text":"","role_description":"summary","subrole":"AXSummary","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"LakyLak","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"added","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"commits","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"yesterday","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"yesterday","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@LakyLak","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"JY-20915","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"add alias for EU","depth":14,"on_screen":false,"help_text":"JY-20915 add alias for EU","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"add alias for EU","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"8 / 10 checks OK","depth":14,"on_screen":false,"help_text":"","role_description":"summary","subrole":"AXSummary","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"e5255d5","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"e5255d5","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@LakyLak","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"JY-20915","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"fix env recognization","depth":14,"on_screen":false,"help_text":"JY-20915 fix env recognization","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"fix env recognization","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"11 / 12 checks OK","depth":14,"on_screen":false,"help_text":"","role_description":"summary","subrole":"AXSummary","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"8953054","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"8953054","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@LakyLak","depth":12,"bounds":{"left":0.5249335,"top":0.0,"width":0.0066489363,"height":0.015961692},"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"JY-20915","depth":14,"bounds":{"left":0.53357714,"top":0.0,"width":0.019115692,"height":0.011572227},"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915","depth":15,"bounds":{"left":0.53357714,"top":0.0,"width":0.019115692,"height":0.011572227},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"code smells & tests","depth":14,"bounds":{"left":0.55502,"top":0.0,"width":0.045711435,"height":0.011572227},"on_screen":false,"help_text":"JY-20915 code smells & tests","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"code smells & tests","depth":15,"bounds":{"left":0.55502,"top":0.0,"width":0.045711435,"height":0.011572227},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"11 / 12 checks OK","depth":14,"bounds":{"left":0.7563165,"top":0.0,"width":0.005319149,"height":0.016759777},"on_screen":false,"help_text":"","role_description":"summary","subrole":"AXSummary","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"73d4bea","depth":14,"bounds":{"left":0.76296544,"top":0.0,"width":0.016954787,"height":0.011572227},"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"73d4bea","depth":15,"bounds":{"left":0.76296544,"top":0.0,"width":0.016954787,"height":0.011572227},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@LakyLak","depth":12,"bounds":{"left":0.5249335,"top":0.0,"width":0.0066489363,"height":0.015961692},"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"JY-20915","depth":14,"bounds":{"left":0.53357714,"top":0.0,"width":0.019115692,"height":0.011572227},"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915","depth":15,"bounds":{"left":0.53357714,"top":0.0,"width":0.019115692,"height":0.011572227},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"add more tests","depth":14,"bounds":{"left":0.55502,"top":0.0,"width":0.03357713,"height":0.011572227},"on_screen":false,"help_text":"JY-20915 add more tests","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"add more tests","depth":15,"bounds":{"left":0.55502,"top":0.0,"width":0.03357713,"height":0.011572227},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"11 / 12 checks OK","depth":14,"bounds":{"left":0.7563165,"top":0.0,"width":0.005319149,"height":0.016759777},"on_screen":false,"help_text":"","role_description":"summary","subrole":"AXSummary","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"5d2dd3e","depth":14,"bounds":{"left":0.76296544,"top":0.0,"width":0.016954787,"height":0.011572227},"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"5d2dd3e","depth":15,"bounds":{"left":0.76296544,"top":0.0,"width":0.016954787,"height":0.011572227},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@LakyLak","depth":12,"bounds":{"left":0.5249335,"top":0.0,"width":0.0066489363,"height":0.015961692},"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"JY-20915","depth":14,"bounds":{"left":0.53357714,"top":0.0,"width":0.019115692,"height":0.011572227},"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915","depth":15,"bounds":{"left":0.53357714,"top":0.0,"width":0.019115692,"height":0.011572227},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"add sms received test","depth":14,"bounds":{"left":0.55502,"top":0.0,"width":0.050531916,"height":0.011572227},"on_screen":false,"help_text":"JY-20915 add sms received test","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"add sms received test","depth":15,"bounds":{"left":0.55502,"top":0.0,"width":0.050531916,"height":0.011572227},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"12 / 12 checks OK","depth":14,"bounds":{"left":0.7563165,"top":0.0,"width":0.005319149,"height":0.016759777},"on_screen":false,"help_text":"","role_description":"summary","subrole":"AXSummary","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"b8c8eb5","depth":14,"bounds":{"left":0.76296544,"top":0.0,"width":0.016954787,"height":0.011572227},"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"b8c8eb5","depth":15,"bounds":{"left":0.76296544,"top":0.0,"width":0.016954787,"height":0.011572227},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@sonarqubecloud","depth":13,"bounds":{"left":0.49268618,"top":0.019952115,"width":0.013297873,"height":0.031923383},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show options","depth":14,"bounds":{"left":0.7662899,"top":0.0207502,"width":0.007978723,"height":0.02952913},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXHeading","text":"sonarqubecloud Bot commented 3 hours ago","depth":13,"bounds":{"left":0.5169548,"top":0.0207502,"width":0.24135639,"height":0.029928172},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXLink","text":"sonarqubecloud","depth":15,"bounds":{"left":0.5169548,"top":0.028731046,"width":0.036236703,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"sonarqubecloud","depth":16,"bounds":{"left":0.5169548,"top":0.028731046,"width":0.036236703,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Bot","depth":16,"bounds":{"left":0.55668217,"top":0.030327214,"width":0.0066489363,"height":0.011971269},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"commented","depth":14,"bounds":{"left":0.56698805,"top":0.029130088,"width":0.025598405,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"3 hours ago","depth":14,"bounds":{"left":0.59391624,"top":0.027533919,"width":0.025598405,"height":0.016759777},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false}]...
|
3020352738380213495
|
-5678133564466170088
|
idle
|
accessibility
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo
Share & export
Copy
Show more options
Ask Gemini
Ask Gemini
Upload & tools
Microphone
Send message
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Summarize page
Summarize page
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
All issues(g then i)
All pull requests
All repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Pull requests (32)
Pull requests
(
32
)
Agents
Agents
Actions
Actions
Wiki
Wiki
Security and quality
Security and quality
Insights
Insights
Settings
Settings
JY-20915 add alias for EU #12105 Edit title
JY-20915 add alias for EU
#
12105
Edit title
Awaiting approval
Awaiting approval
Code
Code
Open
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Lines changed: 365 additions & 1 deletion
Conversation (1)
Conversation
(
1
)
Commits (5)
Commits
(
5
)
Checks (3)
Checks
(
3
)
Files changed (4)
Files changed
(
4
)
Open
JY-20915 add alias for EU #12105 LakyLak wants to merge 5 commits into master from JY-20915-add-domain-specific-email-text-relay Copy head branch name to clipboard
JY-20915 add alias for EU
JY-20915 add alias for EU
#
12105
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Conversation
Conversation
@LakyLak
Show options
LakyLak commented yesterday
LakyLak
LakyLak
commented
yesterday
yesterday
JIRA: JY-20915
JIRA:
JY-20915
JY-20915
Changes:
Changes:
Support text relay for EU
Prevent errors from EU on US
Add or remove reactions
LakyLak
LakyLak
added
5
commits
yesterday
yesterday
@LakyLak
JY-20915
JY-20915
add alias for EU
add alias for EU
8 / 10 checks OK
e5255d5
e5255d5
@LakyLak
JY-20915
JY-20915
fix env recognization
fix env recognization
11 / 12 checks OK
8953054
8953054
@LakyLak
JY-20915
JY-20915
code smells & tests
code smells & tests
11 / 12 checks OK
73d4bea
73d4bea
@LakyLak
JY-20915
JY-20915
add more tests
add more tests
11 / 12 checks OK
5d2dd3e
5d2dd3e
@LakyLak
JY-20915
JY-20915
add sms received test
add sms received test
12 / 12 checks OK
b8c8eb5
b8c8eb5
@sonarqubecloud
Show options
sonarqubecloud Bot commented 3 hours ago
sonarqubecloud
sonarqubecloud
Bot
commented
3 hours ago...
|
NULL
|
NULL
|
NULL
|
NULL
|
|
65243
|
2287
|
5
|
2026-05-20T15:32:48.642283+00:00
|
/Users/lukas/.screenpipe/data/data/2026-05-20/1779 /Users/lukas/.screenpipe/data/data/2026-05-20/1779291168642_m1.jpg...
|
Firefox
|
JY-20915 add alias for EU by LakyLak · Pull Reques JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app — Work...
|
1
|
github.com/jiminny/app/pull/12105
|
monitor_1
|
NULL
|
NULL
|
NULL
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or...
|
[{"role":"AXRadioButton","text [{"role":"AXRadioButton","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Close tab","depth":5,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXRadioButton","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Pipelines - jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Pipelines - jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Tab","depth":4,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Customize sidebar","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Close Google Gemini (⌃X)","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Tabs from other devices","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open history (⇧⌘H)","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open bookmarks (⌘B)","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"AI Chat settings","depth":7,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Close","depth":7,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Main menu","depth":13,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"New chat","depth":12,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Gemini","depth":15,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Chat","depth":13,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Open menu for conversation actions.","depth":13,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Conversation with Gemini","depth":15,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Conversation with Gemini","depth":16,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"You said I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.","depth":23,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Expand","depth":21,"on_screen":false,"role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a concise summary of the changes made in the code selection:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Refactoring and Dependencies","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Refactoring and Dependencies","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Replaced Interface with Implementation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Changed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusInterface","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusManager","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"in the imports and constructor dependency injection.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Redis Support:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Imported","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Illuminate\\Support\\Facades\\Redis","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to handle a newly introduced memory-logging lock mechanism.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Traits:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"WorkerIdTrait","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", along with the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"worker","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"field tracking in the final logs.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Loop and Worker Lifecycle Adjustments","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Loop and Worker Lifecycle Adjustments","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Worker Auto-Restart:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Deleted the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property, the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"constant, and the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"shouldIterateAgain()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method. The script no longer tracks execution iterations or triggers a self-managed worker restart.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Simplified Chunk Method Return:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Modified","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to always return","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Priority Handling for Throttled Entities","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Priority Handling for Throttled Entities","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Stale Record Elevation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Updated the logic in","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"rescheduleThrottledEntities()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to check if a throttled entity group is classified as \"stale\". If it is, the entities are rescheduled using","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"QueuePriorityEnum::NORMAL","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"; otherwise, they keep their respective queue priority.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Metrics, Logging, and Environment Handling","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Metrics, Logging, and Environment Handling","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Environment Tracking:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Introduced a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$environment","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property (defaulting to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"'local'","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") and a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"setEnvironment()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method to dynamically assign it.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Memory Monitoring via Datadog Gauge:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method that submits peak memory allocation to Datadog as a gauge metric (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"jiminny.reindex-memory-usage","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis Lock for Metrics Throttling:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Implemented a Redis-backed lock (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"%s-process-manager-lock","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") with a 60-second expiration inside","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to prevent over-reporting metrics.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Log Payload Update:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added total processed items (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$updatedDocuments + $deletedDocuments","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") to the final information log.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":23,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"You said are there any issues or potential bottlenecks","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"are there any issues or potential bottlenecks","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Edit","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a breakdown of the potential issues to look out for:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"1. Memory Leak Risk (OOM Crashes)","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"1. Memory Leak Risk (OOM Crashes)","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Restart Logic:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The most significant change is the removal of the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Dependency Leaks:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If any underlying service (like the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ElasticSearchClient","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", ORM, or Datadog helper) retains references to processed entities,","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Hardcoded Limits:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The code still includes","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ini_set('memory_limit', '512M');","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":". Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"2. Race Condition in Redis Lock","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"2. Race Condition in Redis Lock","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Non-Atomic Operations:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method uses three distinct steps to handle the lock:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"get()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":",","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", and","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"expire()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If multiple workers process the same","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$this->updateTarget","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"concurrently, they can both pass the","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"if (! is_null(Redis::get(...)))","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"check at the exact same millisecond. Both will then execute","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and push duplicate metrics to Datadog.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Fix:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"This should be replaced with a single atomic operation using Redis's","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"NX","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Not eXists) and","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"EX","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Expire) flags, or Laravel’s built-in atomic cache locks:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis::set($gaugeLockName, true, 'EX', 60, 'NX')","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"3. Infinite Loops and Caller Contract","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"3. Infinite Loops and Caller Contract","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Return Value Change:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"previously returned whether it should iterate again based on staleness or chunk size. It now always returns","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(unless the selection is totally empty).","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the code calling this method relies on a","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"false","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"return to pause, sleep, or gracefully terminate, changing this to an unconditional","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"could result in an aggressive \"busy loop\" that spikes CPU usage.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"4. Performance Overhead","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"4. Performance Overhead","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Expensive Garbage Collection:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Calling","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles();","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"on","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"every single chunk","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Synchronous Metric Dispatch:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::increment","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"or","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"}]...
|
4903914554216228575
|
-1079959478791211044
|
idle
|
accessibility
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or...
|
NULL
|
NULL
|
NULL
|
NULL
|
|
65244
|
2288
|
5
|
2026-05-20T15:33:00.353051+00:00
|
/Users/lukas/.screenpipe/data/data/2026-05-20/1779 /Users/lukas/.screenpipe/data/data/2026-05-20/1779291180353_m2.jpg...
|
Firefox
|
JY-20915 add alias for EU by LakyLak · Pull Reques JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app — Work...
|
1
|
github.com/jiminny/app/pull/12105
|
monitor_2
|
NULL
|
NULL
|
NULL
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo
Share & export
Copy
Show more options
Ask Gemini
Ask Gemini
Upload & tools
Microphone
Send message
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Summarize page
Summarize page
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
All issues(g then i)
All pull requests
All repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Pull requests (32)
Pull requests
(
32
)
Agents
Agents
Actions
Actions
Wiki
Wiki
Security and quality
Security and quality
Insights
Insights
Settings
Settings
JY-20915 add alias for EU #12105 Edit title
JY-20915 add alias for EU
#
12105
Edit title
Awaiting approval
Awaiting approval
Code
Code
Open
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Lines changed: 365 additions & 1 deletion
Conversation (1)
Conversation
(
1
)
Commits (5)
Commits
(
5
)
Checks (3)
Checks
(
3
)
Files changed (4)
Files changed
(
4
)
Open
JY-20915 add alias for EU #12105 LakyLak wants to merge 5 commits into master from JY-20915-add-domain-specific-email-text-relay Copy head branch name to clipboard
JY-20915 add alias for EU
JY-20915 add alias for EU
#
12105
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Conversation
Conversation
@LakyLak
Show options
LakyLak commented yesterday
LakyLak
LakyLak
commented
yesterday
yesterday
JIRA: JY-20915
JIRA:
JY-20915
JY-20915
Changes:
Changes:
Support text relay for EU
Prevent errors from EU on US
Add or remove reactions
LakyLak
LakyLak
added
5
commits
yesterday
yesterday
@LakyLak
JY-20915
JY-20915
add alias for EU
add alias for EU...
|
[{"role":"AXRadioButton","text [{"role":"AXRadioButton","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":4,"bounds":{"left":0.0,"top":0.0518755,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.06304868,"width":0.10106383,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":4,"bounds":{"left":0.0,"top":0.08459697,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.09577015,"width":0.10721409,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.11731844,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.12849163,"width":0.15791224,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.15003991,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.16121309,"width":0.1796875,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.18276137,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.19393456,"width":0.12699468,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Close tab","depth":5,"bounds":{"left":0.06732048,"top":0.18994413,"width":0.007978723,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXRadioButton","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":4,"bounds":{"left":0.0,"top":0.21548285,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":5,"bounds":{"left":0.013297873,"top":0.22665602,"width":0.10139628,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":4,"bounds":{"left":0.0,"top":0.2482043,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.25937748,"width":0.18068483,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":4,"bounds":{"left":0.0,"top":0.28092578,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":5,"bounds":{"left":0.013297873,"top":0.29209897,"width":0.4644282,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Pipelines - jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.31364724,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Pipelines - jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.32482043,"width":0.039228722,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.3463687,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.3575419,"width":0.15824468,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Tab","depth":4,"bounds":{"left":0.0028257978,"top":0.38068634,"width":0.07413564,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Customize sidebar","depth":6,"bounds":{"left":0.0028257978,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Close Google Gemini (⌃X)","depth":6,"bounds":{"left":0.013796543,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Tabs from other devices","depth":6,"bounds":{"left":0.024933511,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open history (⇧⌘H)","depth":6,"bounds":{"left":0.036070477,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open bookmarks (⌘B)","depth":6,"bounds":{"left":0.04720745,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"AI Chat settings","depth":7,"bounds":{"left":0.35854387,"top":0.055067837,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Close","depth":7,"bounds":{"left":0.37051198,"top":0.055067837,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Main menu","depth":13,"bounds":{"left":0.08228058,"top":0.09736632,"width":0.015957447,"height":0.03830806},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"New chat","depth":12,"bounds":{"left":0.09823803,"top":0.10215483,"width":0.028091755,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Gemini","depth":15,"bounds":{"left":0.1008976,"top":0.10654429,"width":0.02144282,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Chat","depth":13,"bounds":{"left":0.35854387,"top":0.10215483,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Open menu for conversation actions.","depth":13,"bounds":{"left":0.37051198,"top":0.10215483,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Conversation with Gemini","depth":15,"bounds":{"left":0.079288565,"top":0.14445332,"width":0.0003324468,"height":0.0007980846},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Conversation with Gemini","depth":16,"bounds":{"left":0.079288565,"top":0.14684756,"width":0.1200133,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"You said I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.","depth":23,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Expand","depth":21,"on_screen":false,"role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a concise summary of the changes made in the code selection:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Refactoring and Dependencies","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Refactoring and Dependencies","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Replaced Interface with Implementation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Changed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusInterface","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusManager","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"in the imports and constructor dependency injection.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Redis Support:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Imported","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Illuminate\\Support\\Facades\\Redis","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to handle a newly introduced memory-logging lock mechanism.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Traits:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"WorkerIdTrait","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", along with the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"worker","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"field tracking in the final logs.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Loop and Worker Lifecycle Adjustments","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Loop and Worker Lifecycle Adjustments","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Worker Auto-Restart:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Deleted the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property, the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"constant, and the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"shouldIterateAgain()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method. The script no longer tracks execution iterations or triggers a self-managed worker restart.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Simplified Chunk Method Return:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Modified","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to always return","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Priority Handling for Throttled Entities","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Priority Handling for Throttled Entities","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Stale Record Elevation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Updated the logic in","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"rescheduleThrottledEntities()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to check if a throttled entity group is classified as \"stale\". If it is, the entities are rescheduled using","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"QueuePriorityEnum::NORMAL","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"; otherwise, they keep their respective queue priority.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Metrics, Logging, and Environment Handling","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Metrics, Logging, and Environment Handling","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Environment Tracking:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Introduced a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$environment","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property (defaulting to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"'local'","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") and a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"setEnvironment()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method to dynamically assign it.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Memory Monitoring via Datadog Gauge:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method that submits peak memory allocation to Datadog as a gauge metric (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"jiminny.reindex-memory-usage","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis Lock for Metrics Throttling:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Implemented a Redis-backed lock (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"%s-process-manager-lock","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") with a 60-second expiration inside","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to prevent over-reporting metrics.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Log Payload Update:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added total processed items (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$updatedDocuments + $deletedDocuments","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") to the final information log.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":23,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"You said are there any issues or potential bottlenecks","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"are there any issues or potential bottlenecks","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Edit","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a breakdown of the potential issues to look out for:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"1. Memory Leak Risk (OOM Crashes)","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"1. Memory Leak Risk (OOM Crashes)","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Restart Logic:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The most significant change is the removal of the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Dependency Leaks:","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.050033245,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If any underlying service (like the","depth":26,"bounds":{"left":0.17453457,"top":0.0,"width":0.08277926,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ElasticSearchClient","depth":27,"bounds":{"left":0.25930852,"top":0.0,"width":0.056848403,"height":0.015163607},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", ORM, or Datadog helper) retains references to processed entities,","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.21775267,"height":0.03631285},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"bounds":{"left":0.26811835,"top":0.0,"width":0.056848403,"height":0.015163607},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.23138298,"height":0.05546688},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Hardcoded Limits:","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.047041222,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The code still includes","depth":26,"bounds":{"left":0.17154256,"top":0.0,"width":0.05651596,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ini_set('memory_limit', '512M');","depth":27,"bounds":{"left":0.23005319,"top":0.0,"width":0.095744684,"height":0.015163607},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":". Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.23055187,"height":0.05546688},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"2. Race Condition in Redis Lock","depth":23,"bounds":{"left":0.111369684,"top":0.07940942,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"2. Race Condition in Redis Lock","depth":24,"bounds":{"left":0.111369684,"top":0.079010375,"width":0.09125665,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Non-Atomic Operations:","depth":26,"bounds":{"left":0.12450133,"top":0.10614525,"width":0.062333778,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The","depth":26,"bounds":{"left":0.18683511,"top":0.10614525,"width":0.011635638,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage","depth":27,"bounds":{"left":0.20046543,"top":0.10774142,"width":0.041888297,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method uses three distinct steps to handle the lock:","depth":26,"bounds":{"left":0.12450133,"top":0.10614525,"width":0.22722739,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"get()","depth":27,"bounds":{"left":0.14860372,"top":0.12689546,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":",","depth":26,"bounds":{"left":0.16555852,"top":0.12529927,"width":0.002493351,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"bounds":{"left":0.17004654,"top":0.12689546,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", and","depth":26,"bounds":{"left":0.18700133,"top":0.12529927,"width":0.013131649,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"expire()","depth":27,"bounds":{"left":0.20212767,"top":0.12689546,"width":0.023936171,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"bounds":{"left":0.22805852,"top":0.12529927,"width":0.0013297872,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"bounds":{"left":0.12450133,"top":0.15403032,"width":0.040059842,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If multiple workers process the same","depth":26,"bounds":{"left":0.16456117,"top":0.15403032,"width":0.09275266,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$this->updateTarget","depth":27,"bounds":{"left":0.25930852,"top":0.15562649,"width":0.056848403,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"concurrently, they can both pass the","depth":26,"bounds":{"left":0.12450133,"top":0.15403032,"width":0.22855718,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"if (! is_null(Redis::get(...)))","depth":27,"bounds":{"left":0.18351063,"top":0.17478053,"width":0.09275266,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"check at the exact same millisecond. Both will then execute","depth":26,"bounds":{"left":0.12450133,"top":0.17318435,"width":0.21575798,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"bounds":{"left":0.2122673,"top":0.19393456,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and push duplicate metrics to Datadog.","depth":26,"bounds":{"left":0.22922207,"top":0.19233839,"width":0.09823803,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Fix:","depth":26,"bounds":{"left":0.12450133,"top":0.22106944,"width":0.019281914,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"This should be replaced with a single atomic operation using Redis's","depth":26,"bounds":{"left":0.14378324,"top":0.22106944,"width":0.16821809,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"NX","depth":27,"bounds":{"left":0.31399602,"top":0.22266561,"width":0.005984043,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Not eXists) and","depth":26,"bounds":{"left":0.12450133,"top":0.22106944,"width":0.22855718,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"EX","depth":27,"bounds":{"left":0.13696809,"top":0.24181964,"width":0.005984043,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Expire) flags, or Laravel’s built-in atomic cache locks:","depth":26,"bounds":{"left":0.14494681,"top":0.24022347,"width":0.13264628,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis::set($gaugeLockName, true, 'EX', 60, 'NX')","depth":27,"bounds":{"left":0.12649602,"top":0.26097366,"width":0.14361702,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"bounds":{"left":0.27210772,"top":0.25937748,"width":0.0013297872,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"3. Infinite Loops and Caller Contract","depth":23,"bounds":{"left":0.111369684,"top":0.30925778,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"3. Infinite Loops and Caller Contract","depth":24,"bounds":{"left":0.111369684,"top":0.30885875,"width":0.10571808,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Return Value Change:","depth":26,"bounds":{"left":0.12450133,"top":0.33599362,"width":0.055352394,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"bounds":{"left":0.18301196,"top":0.33758977,"width":0.06881649,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"previously returned whether it should iterate again based on staleness or chunk size. It now always returns","depth":26,"bounds":{"left":0.12450133,"top":0.33599362,"width":0.2237367,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"bounds":{"left":0.29438165,"top":0.3567438,"width":0.011968086,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(unless the selection is totally empty).","depth":26,"bounds":{"left":0.12450133,"top":0.35514766,"width":0.2130984,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"bounds":{"left":0.12450133,"top":0.40303272,"width":0.040059842,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the code calling this method relies on a","depth":26,"bounds":{"left":0.16456117,"top":0.40303272,"width":0.10305851,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"false","depth":27,"bounds":{"left":0.26961437,"top":0.4046289,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"return to pause, sleep, or gracefully terminate, changing this to an unconditional","depth":26,"bounds":{"left":0.12450133,"top":0.40303272,"width":0.22589761,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"bounds":{"left":0.26163563,"top":0.4237829,"width":0.011968086,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"could result in an aggressive \"busy loop\" that spikes CPU usage.","depth":26,"bounds":{"left":0.12450133,"top":0.42218676,"width":0.22290559,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"4. Performance Overhead","depth":23,"bounds":{"left":0.111369684,"top":0.49122107,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"4. Performance Overhead","depth":24,"bounds":{"left":0.111369684,"top":0.49082202,"width":0.076130316,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Expensive Garbage Collection:","depth":26,"bounds":{"left":0.12450133,"top":0.5179569,"width":0.078457445,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Calling","depth":26,"bounds":{"left":0.20295878,"top":0.5179569,"width":0.019281914,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles();","depth":27,"bounds":{"left":0.22423537,"top":0.51955307,"width":0.059840426,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"on","depth":26,"bounds":{"left":0.28607047,"top":0.5179569,"width":0.00880984,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"every single chunk","depth":26,"bounds":{"left":0.29488033,"top":0.5179569,"width":0.045545213,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.","depth":26,"bounds":{"left":0.12450133,"top":0.5179569,"width":0.23420878,"height":0.05546688},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Synchronous Metric Dispatch:","depth":26,"bounds":{"left":0.12450133,"top":0.584996,"width":0.07712766,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the","depth":26,"bounds":{"left":0.20162898,"top":0.584996,"width":0.015292553,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::increment","depth":27,"bounds":{"left":0.21891622,"top":0.5865922,"width":0.053856384,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"or","depth":26,"bounds":{"left":0.27476728,"top":0.584996,"width":0.0078125,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::gauge","depth":27,"bounds":{"left":0.28457448,"top":0.5865922,"width":0.041888297,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.","depth":26,"bounds":{"left":0.12450133,"top":0.584996,"width":0.23420878,"height":0.05546688},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"5. Lack of Explicit Error Handling","depth":23,"bounds":{"left":0.111369684,"top":0.67318434,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5. Lack of Explicit Error Handling","depth":24,"bounds":{"left":0.111369684,"top":0.67278534,"width":0.094913565,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"No","depth":26,"bounds":{"left":0.12450133,"top":0.6999202,"width":0.008477394,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"try/catch","depth":27,"bounds":{"left":0.1349734,"top":0.7015164,"width":0.026928192,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Blocks:","depth":26,"bounds":{"left":0.16389628,"top":0.6999202,"width":0.019614361,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The script executes complex network operations (","depth":26,"bounds":{"left":0.18351063,"top":0.6999202,"width":0.12367021,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"loadDocuments","depth":27,"bounds":{"left":0.30917552,"top":0.7015164,"width":0.038896278,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.","depth":26,"bounds":{"left":0.12450133,"top":0.6999202,"width":0.22888963,"height":0.09377494},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":22,"bounds":{"left":0.1087101,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":22,"bounds":{"left":0.124667555,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Redo","depth":22,"bounds":{"left":0.140625,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":21,"bounds":{"left":0.15658244,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":22,"bounds":{"left":0.17253989,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":21,"bounds":{"left":0.18849733,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXTextArea","text":"Ask Gemini","depth":21,"bounds":{"left":0.14594415,"top":0.86632085,"width":0.17021276,"height":0.01915403},"on_screen":true,"value":"Ask Gemini","help_text":"","role_description":"text entry area","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Ask Gemini","depth":22,"bounds":{"left":0.14594415,"top":0.8667199,"width":0.030418882,"height":0.018355945},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Upload & tools","depth":20,"bounds":{"left":0.12732713,"top":0.8567438,"width":0.015957447,"height":0.03830806},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Microphone","depth":20,"bounds":{"left":0.32413563,"top":0.86153233,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Send message","depth":20,"bounds":{"left":0.33211437,"top":0.8631285,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":false,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.","depth":17,"bounds":{"left":0.102726065,"top":0.9173983,"width":0.21775267,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Your privacy & Gemini Opens in a new window","depth":17,"bounds":{"left":0.32047874,"top":0.9173983,"width":0.043882977,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Your privacy & Gemini","depth":18,"bounds":{"left":0.32047874,"top":0.9173983,"width":0.043882977,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Opens in a new window","depth":19,"bounds":{"left":0.079288565,"top":0.91660017,"width":0.04720745,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Summarize page","depth":7,"bounds":{"left":0.08494016,"top":0.95730245,"width":0.053523935,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Summarize page","depth":9,"bounds":{"left":0.09059176,"top":0.96249,"width":0.042220745,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Skip to content","depth":6,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Skip to content","depth":7,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Open menu","depth":10,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"Homepage (g then d)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"jiminny","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"jiminny","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"app","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"app","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Search or jump to…","depth":9,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Type","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"/","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to search","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Chat with Copilot","depth":10,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXMenuButton","text":"Open Copilot…","depth":9,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXMenuButton","text":"Create new...","depth":9,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"All issues(g then i)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"All pull requests","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"All repositories","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"You have unread notifications(g then n)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Open user navigation menu","depth":9,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Repository navigation","depth":9,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Repository navigation","depth":10,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Code","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Code","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Pull requests (32)","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Pull requests","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"32","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Agents","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Agents","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Actions","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Actions","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Wiki","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Wiki","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Security and quality","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Security and quality","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Insights","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Insights","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Settings","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Settings","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"JY-20915 add alias for EU #12105 Edit title","depth":13,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"JY-20915 add alias for EU","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"#","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"12105","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Edit title","depth":14,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Awaiting approval","depth":13,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Awaiting approval","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXMenuButton","text":"Code","depth":13,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXStaticText","text":"Code","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Open","depth":13,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"LakyLak","depth":15,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"wants to merge 5 commits into","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"master","depth":15,"on_screen":false,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"master","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"from","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915-add-domain-specific-email-text-relay","depth":16,"on_screen":false,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915-add-domain-specific-email-text-relay","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy head branch name to clipboard","depth":16,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Lines changed: 365 additions & 1 deletion","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Conversation (1)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"Conversation","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"1","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Commits (5)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Commits","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Checks (3)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Checks","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"3","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Files changed (4)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Files changed","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"4","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Open","depth":14,"bounds":{"left":0.50332445,"top":0.07222666,"width":0.011968086,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"JY-20915 add alias for EU #12105 LakyLak wants to merge 5 commits into master from JY-20915-add-domain-specific-email-text-relay Copy head branch name to clipboard","depth":14,"bounds":{"left":0.5219415,"top":0.058260176,"width":0.22922207,"height":0.042298485},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915 add alias for EU","depth":16,"bounds":{"left":0.5219415,"top":0.05865922,"width":0.05817819,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915 add alias for EU","depth":17,"bounds":{"left":0.5219415,"top":0.06304868,"width":0.05817819,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"#","depth":16,"bounds":{"left":0.5827792,"top":0.06304868,"width":0.0028257978,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"12105","depth":16,"bounds":{"left":0.585605,"top":0.06304868,"width":0.012799202,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"LakyLak","depth":18,"bounds":{"left":0.5219415,"top":0.08339984,"width":0.016123671,"height":0.011971269},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":19,"bounds":{"left":0.5219415,"top":0.08339984,"width":0.016123671,"height":0.011971269},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"wants to merge 5 commits into","depth":18,"bounds":{"left":0.539395,"top":0.08339984,"width":0.058011968,"height":0.011971269},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"master","depth":18,"bounds":{"left":0.5987367,"top":0.08180367,"width":0.018450798,"height":0.015163607},"on_screen":true,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"master","depth":19,"bounds":{"left":0.6007314,"top":0.08339984,"width":0.014461436,"height":0.011572227},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"from","depth":19,"bounds":{"left":0.6185173,"top":0.08339984,"width":0.008643617,"height":0.011971269},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915-add-domain-specific-email-text-relay","depth":19,"bounds":{"left":0.6284907,"top":0.08180367,"width":0.112034574,"height":0.015163607},"on_screen":true,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915-add-domain-specific-email-text-relay","depth":20,"bounds":{"left":0.63048536,"top":0.08339984,"width":0.10804521,"height":0.011572227},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy head branch name to clipboard","depth":19,"bounds":{"left":0.741855,"top":0.07821229,"width":0.00930851,"height":0.022346368},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Conversation","depth":12,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Conversation","depth":13,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@LakyLak","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show options","depth":15,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXHeading","text":"LakyLak commented yesterday","depth":14,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXLink","text":"LakyLak","depth":16,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"commented","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"yesterday","depth":15,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"yesterday","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"JIRA: JY-20915","depth":16,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"JIRA:","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915","depth":17,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Changes:","depth":16,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Changes:","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Support text relay for EU","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Prevent errors from EU on US","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Add or remove reactions","depth":16,"on_screen":false,"help_text":"","role_description":"summary","subrole":"AXSummary","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"LakyLak","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"added","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"commits","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"yesterday","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"yesterday","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@LakyLak","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"JY-20915","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"add alias for EU","depth":14,"on_screen":false,"help_text":"JY-20915 add alias for EU","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"add alias for EU","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"}]...
|
5527032911304237342
|
-1048292410041026792
|
idle
|
accessibility
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo
Share & export
Copy
Show more options
Ask Gemini
Ask Gemini
Upload & tools
Microphone
Send message
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Summarize page
Summarize page
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
All issues(g then i)
All pull requests
All repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Pull requests (32)
Pull requests
(
32
)
Agents
Agents
Actions
Actions
Wiki
Wiki
Security and quality
Security and quality
Insights
Insights
Settings
Settings
JY-20915 add alias for EU #12105 Edit title
JY-20915 add alias for EU
#
12105
Edit title
Awaiting approval
Awaiting approval
Code
Code
Open
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Lines changed: 365 additions & 1 deletion
Conversation (1)
Conversation
(
1
)
Commits (5)
Commits
(
5
)
Checks (3)
Checks
(
3
)
Files changed (4)
Files changed
(
4
)
Open
JY-20915 add alias for EU #12105 LakyLak wants to merge 5 commits into master from JY-20915-add-domain-specific-email-text-relay Copy head branch name to clipboard
JY-20915 add alias for EU
JY-20915 add alias for EU
#
12105
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Conversation
Conversation
@LakyLak
Show options
LakyLak commented yesterday
LakyLak
LakyLak
commented
yesterday
yesterday
JIRA: JY-20915
JIRA:
JY-20915
JY-20915
Changes:
Changes:
Support text relay for EU
Prevent errors from EU on US
Add or remove reactions
LakyLak
LakyLak
added
5
commits
yesterday
yesterday
@LakyLak
JY-20915
JY-20915
add alias for EU
add alias for EU...
|
65242
|
NULL
|
NULL
|
NULL
|
|
65245
|
2287
|
6
|
2026-05-20T15:33:19.012495+00:00
|
/Users/lukas/.screenpipe/data/data/2026-05-20/1779 /Users/lukas/.screenpipe/data/data/2026-05-20/1779291199012_m1.jpg...
|
Firefox
|
JY-20915 add alias for EU by LakyLak · Pull Reques JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app — Work...
|
1
|
github.com/jiminny/app/pull/12105
|
monitor_1
|
NULL
|
NULL
|
NULL
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract...
|
[{"role":"AXRadioButton","text [{"role":"AXRadioButton","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Close tab","depth":5,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXRadioButton","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Pipelines - jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Pipelines - jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Tab","depth":4,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Customize sidebar","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Close Google Gemini (⌃X)","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Tabs from other devices","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open history (⇧⌘H)","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open bookmarks (⌘B)","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"AI Chat settings","depth":7,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Close","depth":7,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Main menu","depth":13,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"New chat","depth":12,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Gemini","depth":15,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Chat","depth":13,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Open menu for conversation actions.","depth":13,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Conversation with Gemini","depth":15,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Conversation with Gemini","depth":16,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"You said I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.","depth":23,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Expand","depth":21,"on_screen":false,"role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a concise summary of the changes made in the code selection:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Refactoring and Dependencies","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Refactoring and Dependencies","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Replaced Interface with Implementation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Changed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusInterface","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusManager","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"in the imports and constructor dependency injection.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Redis Support:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Imported","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Illuminate\\Support\\Facades\\Redis","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to handle a newly introduced memory-logging lock mechanism.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Traits:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"WorkerIdTrait","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", along with the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"worker","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"field tracking in the final logs.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Loop and Worker Lifecycle Adjustments","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Loop and Worker Lifecycle Adjustments","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Worker Auto-Restart:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Deleted the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property, the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"constant, and the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"shouldIterateAgain()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method. The script no longer tracks execution iterations or triggers a self-managed worker restart.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Simplified Chunk Method Return:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Modified","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to always return","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Priority Handling for Throttled Entities","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Priority Handling for Throttled Entities","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Stale Record Elevation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Updated the logic in","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"rescheduleThrottledEntities()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to check if a throttled entity group is classified as \"stale\". If it is, the entities are rescheduled using","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"QueuePriorityEnum::NORMAL","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"; otherwise, they keep their respective queue priority.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Metrics, Logging, and Environment Handling","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Metrics, Logging, and Environment Handling","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Environment Tracking:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Introduced a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$environment","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property (defaulting to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"'local'","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") and a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"setEnvironment()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method to dynamically assign it.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Memory Monitoring via Datadog Gauge:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method that submits peak memory allocation to Datadog as a gauge metric (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"jiminny.reindex-memory-usage","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis Lock for Metrics Throttling:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Implemented a Redis-backed lock (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"%s-process-manager-lock","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") with a 60-second expiration inside","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to prevent over-reporting metrics.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Log Payload Update:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added total processed items (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$updatedDocuments + $deletedDocuments","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") to the final information log.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":23,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"You said are there any issues or potential bottlenecks","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"are there any issues or potential bottlenecks","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Edit","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a breakdown of the potential issues to look out for:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"1. Memory Leak Risk (OOM Crashes)","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"1. Memory Leak Risk (OOM Crashes)","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Restart Logic:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The most significant change is the removal of the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Dependency Leaks:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If any underlying service (like the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ElasticSearchClient","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", ORM, or Datadog helper) retains references to processed entities,","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Hardcoded Limits:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The code still includes","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ini_set('memory_limit', '512M');","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":". Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"2. Race Condition in Redis Lock","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"2. Race Condition in Redis Lock","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Non-Atomic Operations:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method uses three distinct steps to handle the lock:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"get()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":",","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", and","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"expire()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If multiple workers process the same","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$this->updateTarget","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"concurrently, they can both pass the","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"if (! is_null(Redis::get(...)))","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"check at the exact same millisecond. Both will then execute","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and push duplicate metrics to Datadog.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Fix:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"This should be replaced with a single atomic operation using Redis's","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"NX","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Not eXists) and","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"EX","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Expire) flags, or Laravel’s built-in atomic cache locks:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis::set($gaugeLockName, true, 'EX', 60, 'NX')","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"3. Infinite Loops and Caller Contract","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"}]...
|
3365488222343520674
|
-1061874711537616932
|
idle
|
accessibility
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract...
|
65243
|
NULL
|
NULL
|
NULL
|
|
65246
|
2288
|
6
|
2026-05-20T15:33:30.810652+00:00
|
/Users/lukas/.screenpipe/data/data/2026-05-20/1779 /Users/lukas/.screenpipe/data/data/2026-05-20/1779291210810_m2.jpg...
|
Firefox
|
JY-20915 add alias for EU by LakyLak · Pull Reques JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app — Work...
|
1
|
github.com/jiminny/app/pull/12105
|
monitor_2
|
NULL
|
NULL
|
NULL
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo
Share & export
Copy
Show more options
Ask Gemini
Ask Gemini
Upload & tools
Microphone
Send message
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Summarize page
Summarize page
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
All issues(g then i)
All pull requests
All repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Pull requests (32)
Pull requests
(
32
)
Agents
Agents
Actions
Actions
Wiki
Wiki
Security and quality
Security and quality
Insights
Insights
Settings
Settings
JY-20915 add alias for EU #12105 Edit title
JY-20915 add alias for EU
#
12105
Edit title
Awaiting approval
Awaiting approval
Code
Code
Open
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Lines changed: 365 additions & 1 deletion
Conversation (1)
Conversation
(
1
)
Commits (5)
Commits
(
5
)
Checks (3)
Checks
(
3
)
Files changed (4)
Files changed
(
4
)
Open
JY-20915 add alias for EU #12105 LakyLak wants to merge 5 commits into master from JY-20915-add-domain-specific-email-text-relay Copy head branch name to clipboard
JY-20915 add alias for EU
JY-20915 add alias for EU
#
12105
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Conversation
Conversation
@LakyLak
Show options
LakyLak commented yesterday
LakyLak
LakyLak
commented
yesterday
yesterday
JIRA: JY-20915
JIRA:
JY-20915
JY-20915
Changes:
Changes:
Support text relay for EU
Prevent errors from EU on US
Add or remove reactions
LakyLak
LakyLak
added
5
commits
yesterday
yesterday
@LakyLak
JY-20915
JY-20915
add alias for EU
add alias for EU
8 / 10 checks OK
e5255d5
e5255d5
@LakyLak
JY-20915
JY-20915
fix env recognization
fix env recognization
11 / 12 checks OK
8953054
8953054
@LakyLak
JY-20915
JY-20915
code smells & tests
code smells & tests
11 / 12 checks OK
73d4bea
73d4bea
@LakyLak
JY-20915...
|
[{"role":"AXRadioButton","text [{"role":"AXRadioButton","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":4,"bounds":{"left":0.0,"top":0.0518755,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.06304868,"width":0.10106383,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":4,"bounds":{"left":0.0,"top":0.08459697,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.09577015,"width":0.10721409,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.11731844,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.12849163,"width":0.15791224,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.15003991,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.16121309,"width":0.1796875,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.18276137,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.19393456,"width":0.12699468,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Close tab","depth":5,"bounds":{"left":0.06732048,"top":0.18994413,"width":0.007978723,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXRadioButton","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":4,"bounds":{"left":0.0,"top":0.21548285,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":5,"bounds":{"left":0.013297873,"top":0.22665602,"width":0.10139628,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":4,"bounds":{"left":0.0,"top":0.2482043,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.25937748,"width":0.18068483,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":4,"bounds":{"left":0.0,"top":0.28092578,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":5,"bounds":{"left":0.013297873,"top":0.29209897,"width":0.4644282,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Pipelines - jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.31364724,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Pipelines - jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.32482043,"width":0.039228722,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.3463687,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.3575419,"width":0.15824468,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Tab","depth":4,"bounds":{"left":0.0028257978,"top":0.38068634,"width":0.07413564,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Customize sidebar","depth":6,"bounds":{"left":0.0028257978,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Close Google Gemini (⌃X)","depth":6,"bounds":{"left":0.013796543,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Tabs from other devices","depth":6,"bounds":{"left":0.024933511,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open history (⇧⌘H)","depth":6,"bounds":{"left":0.036070477,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open bookmarks (⌘B)","depth":6,"bounds":{"left":0.04720745,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"AI Chat settings","depth":7,"bounds":{"left":0.35854387,"top":0.055067837,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Close","depth":7,"bounds":{"left":0.37051198,"top":0.055067837,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Main menu","depth":13,"bounds":{"left":0.08228058,"top":0.09736632,"width":0.015957447,"height":0.03830806},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"New chat","depth":12,"bounds":{"left":0.09823803,"top":0.10215483,"width":0.028091755,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Gemini","depth":15,"bounds":{"left":0.1008976,"top":0.10654429,"width":0.02144282,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Chat","depth":13,"bounds":{"left":0.35854387,"top":0.10215483,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Open menu for conversation actions.","depth":13,"bounds":{"left":0.37051198,"top":0.10215483,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Conversation with Gemini","depth":15,"bounds":{"left":0.079288565,"top":0.14445332,"width":0.0003324468,"height":0.0007980846},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Conversation with Gemini","depth":16,"bounds":{"left":0.079288565,"top":0.14684756,"width":0.1200133,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"You said I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.","depth":23,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Expand","depth":21,"on_screen":false,"role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a concise summary of the changes made in the code selection:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Refactoring and Dependencies","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Refactoring and Dependencies","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Replaced Interface with Implementation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Changed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusInterface","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusManager","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"in the imports and constructor dependency injection.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Redis Support:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Imported","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Illuminate\\Support\\Facades\\Redis","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to handle a newly introduced memory-logging lock mechanism.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Traits:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"WorkerIdTrait","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", along with the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"worker","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"field tracking in the final logs.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Loop and Worker Lifecycle Adjustments","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Loop and Worker Lifecycle Adjustments","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Worker Auto-Restart:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Deleted the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property, the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"constant, and the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"shouldIterateAgain()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method. The script no longer tracks execution iterations or triggers a self-managed worker restart.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Simplified Chunk Method Return:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Modified","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to always return","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Priority Handling for Throttled Entities","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Priority Handling for Throttled Entities","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Stale Record Elevation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Updated the logic in","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"rescheduleThrottledEntities()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to check if a throttled entity group is classified as \"stale\". If it is, the entities are rescheduled using","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"QueuePriorityEnum::NORMAL","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"; otherwise, they keep their respective queue priority.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Metrics, Logging, and Environment Handling","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Metrics, Logging, and Environment Handling","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Environment Tracking:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Introduced a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$environment","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property (defaulting to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"'local'","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") and a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"setEnvironment()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method to dynamically assign it.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Memory Monitoring via Datadog Gauge:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method that submits peak memory allocation to Datadog as a gauge metric (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"jiminny.reindex-memory-usage","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis Lock for Metrics Throttling:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Implemented a Redis-backed lock (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"%s-process-manager-lock","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") with a 60-second expiration inside","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to prevent over-reporting metrics.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Log Payload Update:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added total processed items (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$updatedDocuments + $deletedDocuments","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") to the final information log.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":23,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"You said are there any issues or potential bottlenecks","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"are there any issues or potential bottlenecks","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Edit","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a breakdown of the potential issues to look out for:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"1. Memory Leak Risk (OOM Crashes)","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"1. Memory Leak Risk (OOM Crashes)","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Restart Logic:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The most significant change is the removal of the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Dependency Leaks:","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.050033245,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If any underlying service (like the","depth":26,"bounds":{"left":0.17453457,"top":0.0,"width":0.08277926,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ElasticSearchClient","depth":27,"bounds":{"left":0.25930852,"top":0.0,"width":0.056848403,"height":0.015163607},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", ORM, or Datadog helper) retains references to processed entities,","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.21775267,"height":0.03631285},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"bounds":{"left":0.26811835,"top":0.0,"width":0.056848403,"height":0.015163607},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.23138298,"height":0.05546688},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Hardcoded Limits:","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.047041222,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The code still includes","depth":26,"bounds":{"left":0.17154256,"top":0.0,"width":0.05651596,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ini_set('memory_limit', '512M');","depth":27,"bounds":{"left":0.23005319,"top":0.0,"width":0.095744684,"height":0.015163607},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":". Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.23055187,"height":0.05546688},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"2. Race Condition in Redis Lock","depth":23,"bounds":{"left":0.111369684,"top":0.07940942,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"2. Race Condition in Redis Lock","depth":24,"bounds":{"left":0.111369684,"top":0.079010375,"width":0.09125665,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Non-Atomic Operations:","depth":26,"bounds":{"left":0.12450133,"top":0.10614525,"width":0.062333778,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The","depth":26,"bounds":{"left":0.18683511,"top":0.10614525,"width":0.011635638,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage","depth":27,"bounds":{"left":0.20046543,"top":0.10774142,"width":0.041888297,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method uses three distinct steps to handle the lock:","depth":26,"bounds":{"left":0.12450133,"top":0.10614525,"width":0.22722739,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"get()","depth":27,"bounds":{"left":0.14860372,"top":0.12689546,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":",","depth":26,"bounds":{"left":0.16555852,"top":0.12529927,"width":0.002493351,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"bounds":{"left":0.17004654,"top":0.12689546,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", and","depth":26,"bounds":{"left":0.18700133,"top":0.12529927,"width":0.013131649,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"expire()","depth":27,"bounds":{"left":0.20212767,"top":0.12689546,"width":0.023936171,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"bounds":{"left":0.22805852,"top":0.12529927,"width":0.0013297872,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"bounds":{"left":0.12450133,"top":0.15403032,"width":0.040059842,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If multiple workers process the same","depth":26,"bounds":{"left":0.16456117,"top":0.15403032,"width":0.09275266,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$this->updateTarget","depth":27,"bounds":{"left":0.25930852,"top":0.15562649,"width":0.056848403,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"concurrently, they can both pass the","depth":26,"bounds":{"left":0.12450133,"top":0.15403032,"width":0.22855718,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"if (! is_null(Redis::get(...)))","depth":27,"bounds":{"left":0.18351063,"top":0.17478053,"width":0.09275266,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"check at the exact same millisecond. Both will then execute","depth":26,"bounds":{"left":0.12450133,"top":0.17318435,"width":0.21575798,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"bounds":{"left":0.2122673,"top":0.19393456,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and push duplicate metrics to Datadog.","depth":26,"bounds":{"left":0.22922207,"top":0.19233839,"width":0.09823803,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Fix:","depth":26,"bounds":{"left":0.12450133,"top":0.22106944,"width":0.019281914,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"This should be replaced with a single atomic operation using Redis's","depth":26,"bounds":{"left":0.14378324,"top":0.22106944,"width":0.16821809,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"NX","depth":27,"bounds":{"left":0.31399602,"top":0.22266561,"width":0.005984043,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Not eXists) and","depth":26,"bounds":{"left":0.12450133,"top":0.22106944,"width":0.22855718,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"EX","depth":27,"bounds":{"left":0.13696809,"top":0.24181964,"width":0.005984043,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Expire) flags, or Laravel’s built-in atomic cache locks:","depth":26,"bounds":{"left":0.14494681,"top":0.24022347,"width":0.13264628,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis::set($gaugeLockName, true, 'EX', 60, 'NX')","depth":27,"bounds":{"left":0.12649602,"top":0.26097366,"width":0.14361702,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"bounds":{"left":0.27210772,"top":0.25937748,"width":0.0013297872,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"3. Infinite Loops and Caller Contract","depth":23,"bounds":{"left":0.111369684,"top":0.30925778,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"3. Infinite Loops and Caller Contract","depth":24,"bounds":{"left":0.111369684,"top":0.30885875,"width":0.10571808,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Return Value Change:","depth":26,"bounds":{"left":0.12450133,"top":0.33599362,"width":0.055352394,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"bounds":{"left":0.18301196,"top":0.33758977,"width":0.06881649,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"previously returned whether it should iterate again based on staleness or chunk size. It now always returns","depth":26,"bounds":{"left":0.12450133,"top":0.33599362,"width":0.2237367,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"bounds":{"left":0.29438165,"top":0.3567438,"width":0.011968086,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(unless the selection is totally empty).","depth":26,"bounds":{"left":0.12450133,"top":0.35514766,"width":0.2130984,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"bounds":{"left":0.12450133,"top":0.40303272,"width":0.040059842,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the code calling this method relies on a","depth":26,"bounds":{"left":0.16456117,"top":0.40303272,"width":0.10305851,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"false","depth":27,"bounds":{"left":0.26961437,"top":0.4046289,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"return to pause, sleep, or gracefully terminate, changing this to an unconditional","depth":26,"bounds":{"left":0.12450133,"top":0.40303272,"width":0.22589761,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"bounds":{"left":0.26163563,"top":0.4237829,"width":0.011968086,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"could result in an aggressive \"busy loop\" that spikes CPU usage.","depth":26,"bounds":{"left":0.12450133,"top":0.42218676,"width":0.22290559,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"4. Performance Overhead","depth":23,"bounds":{"left":0.111369684,"top":0.49122107,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"4. Performance Overhead","depth":24,"bounds":{"left":0.111369684,"top":0.49082202,"width":0.076130316,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Expensive Garbage Collection:","depth":26,"bounds":{"left":0.12450133,"top":0.5179569,"width":0.078457445,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Calling","depth":26,"bounds":{"left":0.20295878,"top":0.5179569,"width":0.019281914,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles();","depth":27,"bounds":{"left":0.22423537,"top":0.51955307,"width":0.059840426,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"on","depth":26,"bounds":{"left":0.28607047,"top":0.5179569,"width":0.00880984,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"every single chunk","depth":26,"bounds":{"left":0.29488033,"top":0.5179569,"width":0.045545213,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.","depth":26,"bounds":{"left":0.12450133,"top":0.5179569,"width":0.23420878,"height":0.05546688},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Synchronous Metric Dispatch:","depth":26,"bounds":{"left":0.12450133,"top":0.584996,"width":0.07712766,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the","depth":26,"bounds":{"left":0.20162898,"top":0.584996,"width":0.015292553,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::increment","depth":27,"bounds":{"left":0.21891622,"top":0.5865922,"width":0.053856384,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"or","depth":26,"bounds":{"left":0.27476728,"top":0.584996,"width":0.0078125,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::gauge","depth":27,"bounds":{"left":0.28457448,"top":0.5865922,"width":0.041888297,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.","depth":26,"bounds":{"left":0.12450133,"top":0.584996,"width":0.23420878,"height":0.05546688},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"5. Lack of Explicit Error Handling","depth":23,"bounds":{"left":0.111369684,"top":0.67318434,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5. Lack of Explicit Error Handling","depth":24,"bounds":{"left":0.111369684,"top":0.67278534,"width":0.094913565,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"No","depth":26,"bounds":{"left":0.12450133,"top":0.6999202,"width":0.008477394,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"try/catch","depth":27,"bounds":{"left":0.1349734,"top":0.7015164,"width":0.026928192,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Blocks:","depth":26,"bounds":{"left":0.16389628,"top":0.6999202,"width":0.019614361,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The script executes complex network operations (","depth":26,"bounds":{"left":0.18351063,"top":0.6999202,"width":0.12367021,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"loadDocuments","depth":27,"bounds":{"left":0.30917552,"top":0.7015164,"width":0.038896278,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.","depth":26,"bounds":{"left":0.12450133,"top":0.6999202,"width":0.22888963,"height":0.09377494},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":22,"bounds":{"left":0.1087101,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":22,"bounds":{"left":0.124667555,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Redo","depth":22,"bounds":{"left":0.140625,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":21,"bounds":{"left":0.15658244,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":22,"bounds":{"left":0.17253989,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":21,"bounds":{"left":0.18849733,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXTextArea","text":"Ask Gemini","depth":21,"bounds":{"left":0.14594415,"top":0.86632085,"width":0.17021276,"height":0.01915403},"on_screen":true,"value":"Ask Gemini","help_text":"","role_description":"text entry area","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Ask Gemini","depth":22,"bounds":{"left":0.14594415,"top":0.8667199,"width":0.030418882,"height":0.018355945},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Upload & tools","depth":20,"bounds":{"left":0.12732713,"top":0.8567438,"width":0.015957447,"height":0.03830806},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Microphone","depth":20,"bounds":{"left":0.32413563,"top":0.86153233,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Send message","depth":20,"bounds":{"left":0.33211437,"top":0.8631285,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":false,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.","depth":17,"bounds":{"left":0.102726065,"top":0.9173983,"width":0.21775267,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Your privacy & Gemini Opens in a new window","depth":17,"bounds":{"left":0.32047874,"top":0.9173983,"width":0.043882977,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Your privacy & Gemini","depth":18,"bounds":{"left":0.32047874,"top":0.9173983,"width":0.043882977,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Opens in a new window","depth":19,"bounds":{"left":0.079288565,"top":0.91660017,"width":0.04720745,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Summarize page","depth":7,"bounds":{"left":0.08494016,"top":0.95730245,"width":0.053523935,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Summarize page","depth":9,"bounds":{"left":0.09059176,"top":0.96249,"width":0.042220745,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Skip to content","depth":6,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Skip to content","depth":7,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Open menu","depth":10,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"Homepage (g then d)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"jiminny","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"jiminny","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"app","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"app","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Search or jump to…","depth":9,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Type","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"/","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to search","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Chat with Copilot","depth":10,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXMenuButton","text":"Open Copilot…","depth":9,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXMenuButton","text":"Create new...","depth":9,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"All issues(g then i)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"All pull requests","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"All repositories","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"You have unread notifications(g then n)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Open user navigation menu","depth":9,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Repository navigation","depth":9,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Repository navigation","depth":10,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Code","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Code","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Pull requests (32)","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Pull requests","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"32","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Agents","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Agents","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Actions","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Actions","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Wiki","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Wiki","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Security and quality","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Security and quality","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Insights","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Insights","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Settings","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Settings","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"JY-20915 add alias for EU #12105 Edit title","depth":13,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"JY-20915 add alias for EU","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"#","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"12105","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Edit title","depth":14,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Awaiting approval","depth":13,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Awaiting approval","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXMenuButton","text":"Code","depth":13,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXStaticText","text":"Code","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Open","depth":13,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"LakyLak","depth":15,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"wants to merge 5 commits into","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"master","depth":15,"on_screen":false,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"master","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"from","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915-add-domain-specific-email-text-relay","depth":16,"on_screen":false,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915-add-domain-specific-email-text-relay","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy head branch name to clipboard","depth":16,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Lines changed: 365 additions & 1 deletion","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Conversation (1)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"Conversation","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"1","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Commits (5)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Commits","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Checks (3)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Checks","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"3","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Files changed (4)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Files changed","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"4","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Open","depth":14,"bounds":{"left":0.50332445,"top":0.07222666,"width":0.011968086,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"JY-20915 add alias for EU #12105 LakyLak wants to merge 5 commits into master from JY-20915-add-domain-specific-email-text-relay Copy head branch name to clipboard","depth":14,"bounds":{"left":0.5219415,"top":0.058260176,"width":0.22922207,"height":0.042298485},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915 add alias for EU","depth":16,"bounds":{"left":0.5219415,"top":0.05865922,"width":0.05817819,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915 add alias for EU","depth":17,"bounds":{"left":0.5219415,"top":0.06304868,"width":0.05817819,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"#","depth":16,"bounds":{"left":0.5827792,"top":0.06304868,"width":0.0028257978,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"12105","depth":16,"bounds":{"left":0.585605,"top":0.06304868,"width":0.012799202,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"LakyLak","depth":18,"bounds":{"left":0.5219415,"top":0.08339984,"width":0.016123671,"height":0.011971269},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":19,"bounds":{"left":0.5219415,"top":0.08339984,"width":0.016123671,"height":0.011971269},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"wants to merge 5 commits into","depth":18,"bounds":{"left":0.539395,"top":0.08339984,"width":0.058011968,"height":0.011971269},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"master","depth":18,"bounds":{"left":0.5987367,"top":0.08180367,"width":0.018450798,"height":0.015163607},"on_screen":true,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"master","depth":19,"bounds":{"left":0.6007314,"top":0.08339984,"width":0.014461436,"height":0.011572227},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"from","depth":19,"bounds":{"left":0.6185173,"top":0.08339984,"width":0.008643617,"height":0.011971269},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915-add-domain-specific-email-text-relay","depth":19,"bounds":{"left":0.6284907,"top":0.08180367,"width":0.112034574,"height":0.015163607},"on_screen":true,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915-add-domain-specific-email-text-relay","depth":20,"bounds":{"left":0.63048536,"top":0.08339984,"width":0.10804521,"height":0.011572227},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy head branch name to clipboard","depth":19,"bounds":{"left":0.741855,"top":0.07821229,"width":0.00930851,"height":0.022346368},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Conversation","depth":12,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Conversation","depth":13,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@LakyLak","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show options","depth":15,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXHeading","text":"LakyLak commented yesterday","depth":14,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXLink","text":"LakyLak","depth":16,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"commented","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"yesterday","depth":15,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"yesterday","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"JIRA: JY-20915","depth":16,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"JIRA:","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915","depth":17,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Changes:","depth":16,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Changes:","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Support text relay for EU","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Prevent errors from EU on US","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Add or remove reactions","depth":16,"on_screen":false,"help_text":"","role_description":"summary","subrole":"AXSummary","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"LakyLak","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"added","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"commits","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"yesterday","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"yesterday","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@LakyLak","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"JY-20915","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"add alias for EU","depth":14,"on_screen":false,"help_text":"JY-20915 add alias for EU","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"add alias for EU","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"8 / 10 checks OK","depth":14,"on_screen":false,"help_text":"","role_description":"summary","subrole":"AXSummary","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"e5255d5","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"e5255d5","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@LakyLak","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"JY-20915","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"fix env recognization","depth":14,"on_screen":false,"help_text":"JY-20915 fix env recognization","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"fix env recognization","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"11 / 12 checks OK","depth":14,"on_screen":false,"help_text":"","role_description":"summary","subrole":"AXSummary","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"8953054","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"8953054","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@LakyLak","depth":12,"bounds":{"left":0.5249335,"top":0.0,"width":0.0066489363,"height":0.015961692},"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"JY-20915","depth":14,"bounds":{"left":0.53357714,"top":0.0,"width":0.019115692,"height":0.011572227},"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915","depth":15,"bounds":{"left":0.53357714,"top":0.0,"width":0.019115692,"height":0.011572227},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"code smells & tests","depth":14,"bounds":{"left":0.55502,"top":0.0,"width":0.045711435,"height":0.011572227},"on_screen":false,"help_text":"JY-20915 code smells & tests","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"code smells & tests","depth":15,"bounds":{"left":0.55502,"top":0.0,"width":0.045711435,"height":0.011572227},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"11 / 12 checks OK","depth":14,"bounds":{"left":0.7563165,"top":0.0,"width":0.005319149,"height":0.016759777},"on_screen":false,"help_text":"","role_description":"summary","subrole":"AXSummary","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"73d4bea","depth":14,"bounds":{"left":0.76296544,"top":0.0,"width":0.016954787,"height":0.011572227},"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"73d4bea","depth":15,"bounds":{"left":0.76296544,"top":0.0,"width":0.016954787,"height":0.011572227},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@LakyLak","depth":12,"bounds":{"left":0.5249335,"top":0.0,"width":0.0066489363,"height":0.015961692},"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"JY-20915","depth":14,"bounds":{"left":0.53357714,"top":0.0,"width":0.019115692,"height":0.011572227},"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false}]...
|
1706039854844697031
|
-1075315107316877544
|
idle
|
accessibility
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo
Share & export
Copy
Show more options
Ask Gemini
Ask Gemini
Upload & tools
Microphone
Send message
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Summarize page
Summarize page
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
All issues(g then i)
All pull requests
All repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Pull requests (32)
Pull requests
(
32
)
Agents
Agents
Actions
Actions
Wiki
Wiki
Security and quality
Security and quality
Insights
Insights
Settings
Settings
JY-20915 add alias for EU #12105 Edit title
JY-20915 add alias for EU
#
12105
Edit title
Awaiting approval
Awaiting approval
Code
Code
Open
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Lines changed: 365 additions & 1 deletion
Conversation (1)
Conversation
(
1
)
Commits (5)
Commits
(
5
)
Checks (3)
Checks
(
3
)
Files changed (4)
Files changed
(
4
)
Open
JY-20915 add alias for EU #12105 LakyLak wants to merge 5 commits into master from JY-20915-add-domain-specific-email-text-relay Copy head branch name to clipboard
JY-20915 add alias for EU
JY-20915 add alias for EU
#
12105
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Conversation
Conversation
@LakyLak
Show options
LakyLak commented yesterday
LakyLak
LakyLak
commented
yesterday
yesterday
JIRA: JY-20915
JIRA:
JY-20915
JY-20915
Changes:
Changes:
Support text relay for EU
Prevent errors from EU on US
Add or remove reactions
LakyLak
LakyLak
added
5
commits
yesterday
yesterday
@LakyLak
JY-20915
JY-20915
add alias for EU
add alias for EU
8 / 10 checks OK
e5255d5
e5255d5
@LakyLak
JY-20915
JY-20915
fix env recognization
fix env recognization
11 / 12 checks OK
8953054
8953054
@LakyLak
JY-20915
JY-20915
code smells & tests
code smells & tests
11 / 12 checks OK
73d4bea
73d4bea
@LakyLak
JY-20915...
|
NULL
|
NULL
|
NULL
|
NULL
|
|
65247
|
2287
|
7
|
2026-05-20T15:33:49.379626+00:00
|
/Users/lukas/.screenpipe/data/data/2026-05-20/1779 /Users/lukas/.screenpipe/data/data/2026-05-20/1779291229379_m1.jpg...
|
Firefox
|
JY-20915 add alias for EU by LakyLak · Pull Reques JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app — Work...
|
1
|
github.com/jiminny/app/pull/12105
|
monitor_1
|
NULL
|
NULL
|
NULL
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo
Share & export
Copy
Show more options
Ask Gemini
Ask Gemini
Upload & tools
Microphone
Send message
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Summarize page
Summarize page
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
All issues(g then i)
All pull requests
All repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Pull requests (32)
Pull requests
(
32
)
Agents
Agents
Actions
Actions
Wiki
Wiki
Security and quality
Security and quality
Insights
Insights
Settings
Settings
JY-20915 add alias for EU #12105 Edit title
JY-20915 add alias for EU
#
12105
Edit title
Awaiting approval
Awaiting approval
Code
Code
Open
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Lines changed: 365 additions & 1 deletion
Conversation (1)
Conversation...
|
[{"role":"AXRadioButton","text [{"role":"AXRadioButton","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Close tab","depth":5,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXRadioButton","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Pipelines - jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Pipelines - jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Tab","depth":4,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Customize sidebar","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Close Google Gemini (⌃X)","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Tabs from other devices","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open history (⇧⌘H)","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open bookmarks (⌘B)","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"AI Chat settings","depth":7,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Close","depth":7,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Main menu","depth":13,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"New chat","depth":12,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Gemini","depth":15,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Chat","depth":13,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Open menu for conversation actions.","depth":13,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Conversation with Gemini","depth":15,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Conversation with Gemini","depth":16,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"You said I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.","depth":23,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Expand","depth":21,"on_screen":false,"role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a concise summary of the changes made in the code selection:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Refactoring and Dependencies","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Refactoring and Dependencies","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Replaced Interface with Implementation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Changed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusInterface","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusManager","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"in the imports and constructor dependency injection.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Redis Support:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Imported","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Illuminate\\Support\\Facades\\Redis","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to handle a newly introduced memory-logging lock mechanism.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Traits:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"WorkerIdTrait","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", along with the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"worker","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"field tracking in the final logs.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Loop and Worker Lifecycle Adjustments","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Loop and Worker Lifecycle Adjustments","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Worker Auto-Restart:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Deleted the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property, the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"constant, and the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"shouldIterateAgain()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method. The script no longer tracks execution iterations or triggers a self-managed worker restart.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Simplified Chunk Method Return:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Modified","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to always return","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Priority Handling for Throttled Entities","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Priority Handling for Throttled Entities","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Stale Record Elevation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Updated the logic in","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"rescheduleThrottledEntities()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to check if a throttled entity group is classified as \"stale\". If it is, the entities are rescheduled using","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"QueuePriorityEnum::NORMAL","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"; otherwise, they keep their respective queue priority.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Metrics, Logging, and Environment Handling","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Metrics, Logging, and Environment Handling","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Environment Tracking:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Introduced a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$environment","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property (defaulting to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"'local'","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") and a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"setEnvironment()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method to dynamically assign it.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Memory Monitoring via Datadog Gauge:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method that submits peak memory allocation to Datadog as a gauge metric (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"jiminny.reindex-memory-usage","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis Lock for Metrics Throttling:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Implemented a Redis-backed lock (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"%s-process-manager-lock","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") with a 60-second expiration inside","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to prevent over-reporting metrics.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Log Payload Update:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added total processed items (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$updatedDocuments + $deletedDocuments","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") to the final information log.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":23,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"You said are there any issues or potential bottlenecks","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"are there any issues or potential bottlenecks","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Edit","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a breakdown of the potential issues to look out for:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"1. Memory Leak Risk (OOM Crashes)","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"1. Memory Leak Risk (OOM Crashes)","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Restart Logic:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The most significant change is the removal of the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Dependency Leaks:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If any underlying service (like the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ElasticSearchClient","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", ORM, or Datadog helper) retains references to processed entities,","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Hardcoded Limits:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The code still includes","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ini_set('memory_limit', '512M');","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":". Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"2. Race Condition in Redis Lock","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"2. Race Condition in Redis Lock","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Non-Atomic Operations:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method uses three distinct steps to handle the lock:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"get()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":",","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", and","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"expire()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If multiple workers process the same","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$this->updateTarget","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"concurrently, they can both pass the","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"if (! is_null(Redis::get(...)))","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"check at the exact same millisecond. Both will then execute","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and push duplicate metrics to Datadog.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Fix:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"This should be replaced with a single atomic operation using Redis's","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"NX","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Not eXists) and","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"EX","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Expire) flags, or Laravel’s built-in atomic cache locks:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis::set($gaugeLockName, true, 'EX', 60, 'NX')","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"3. Infinite Loops and Caller Contract","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"3. Infinite Loops and Caller Contract","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Return Value Change:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"previously returned whether it should iterate again based on staleness or chunk size. It now always returns","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(unless the selection is totally empty).","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the code calling this method relies on a","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"false","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"return to pause, sleep, or gracefully terminate, changing this to an unconditional","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"could result in an aggressive \"busy loop\" that spikes CPU usage.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"4. Performance Overhead","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"4. Performance Overhead","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Expensive Garbage Collection:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Calling","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles();","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"on","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"every single chunk","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Synchronous Metric Dispatch:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::increment","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"or","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::gauge","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"5. Lack of Explicit Error Handling","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5. Lack of Explicit Error Handling","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"No","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"try/catch","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Blocks:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The script executes complex network operations (","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"loadDocuments","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":22,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":22,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Redo","depth":22,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":21,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":22,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":21,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXTextArea","text":"Ask Gemini","depth":21,"on_screen":true,"value":"Ask Gemini","help_text":"","role_description":"text entry area","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Ask Gemini","depth":22,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Upload & tools","depth":20,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Microphone","depth":20,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Send message","depth":20,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":false,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.","depth":17,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Your privacy & Gemini Opens in a new window","depth":17,"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Your privacy & Gemini","depth":18,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Opens in a new window","depth":19,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Summarize page","depth":7,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Summarize page","depth":9,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Skip to content","depth":6,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Skip to content","depth":7,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Open menu","depth":10,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"Homepage (g then d)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"jiminny","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"jiminny","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"app","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"app","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Search or jump to…","depth":9,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Type","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"/","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to search","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Chat with Copilot","depth":10,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXMenuButton","text":"Open Copilot…","depth":9,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXMenuButton","text":"Create new...","depth":9,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"All issues(g then i)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"All pull requests","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"All repositories","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"You have unread notifications(g then n)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Open user navigation menu","depth":9,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Repository navigation","depth":9,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Repository navigation","depth":10,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Code","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Code","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Pull requests (32)","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Pull requests","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"32","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Agents","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Agents","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Actions","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Actions","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Wiki","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Wiki","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Security and quality","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Security and quality","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Insights","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Insights","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Settings","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Settings","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"JY-20915 add alias for EU #12105 Edit title","depth":13,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"JY-20915 add alias for EU","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"#","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"12105","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Edit title","depth":14,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Awaiting approval","depth":13,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Awaiting approval","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXMenuButton","text":"Code","depth":13,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXStaticText","text":"Code","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Open","depth":13,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"LakyLak","depth":15,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"wants to merge 5 commits into","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"master","depth":15,"on_screen":false,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"master","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"from","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915-add-domain-specific-email-text-relay","depth":16,"on_screen":false,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915-add-domain-specific-email-text-relay","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy head branch name to clipboard","depth":16,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Lines changed: 365 additions & 1 deletion","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Conversation (1)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"Conversation","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"}]...
|
8909552696276690550
|
-1052796009671526500
|
idle
|
accessibility
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo
Share & export
Copy
Show more options
Ask Gemini
Ask Gemini
Upload & tools
Microphone
Send message
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Summarize page
Summarize page
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
All issues(g then i)
All pull requests
All repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Pull requests (32)
Pull requests
(
32
)
Agents
Agents
Actions
Actions
Wiki
Wiki
Security and quality
Security and quality
Insights
Insights
Settings
Settings
JY-20915 add alias for EU #12105 Edit title
JY-20915 add alias for EU
#
12105
Edit title
Awaiting approval
Awaiting approval
Code
Code
Open
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Lines changed: 365 additions & 1 deletion
Conversation (1)
Conversation...
|
NULL
|
NULL
|
NULL
|
NULL
|
|
65248
|
2288
|
7
|
2026-05-20T15:34:01.220566+00:00
|
/Users/lukas/.screenpipe/data/data/2026-05-20/1779 /Users/lukas/.screenpipe/data/data/2026-05-20/1779291241220_m2.jpg...
|
Firefox
|
JY-20915 add alias for EU by LakyLak · Pull Reques JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app — Work...
|
1
|
github.com/jiminny/app/pull/12105
|
monitor_2
|
NULL
|
NULL
|
NULL
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo
Share & export
Copy
Show more options
Ask Gemini
Ask Gemini
Upload & tools
Microphone
Send message
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Summarize page
Summarize page
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
All issues(g then i)
All pull requests
All repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Pull requests (32)
Pull requests
(
32
)
Agents
Agents
Actions
Actions
Wiki
Wiki
Security and quality
Security and quality
Insights
Insights
Settings
Settings
JY-20915 add alias for EU #12105 Edit title
JY-20915 add alias for EU
#
12105
Edit title
Awaiting approval
Awaiting approval
Code
Code
Open
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Lines changed: 365 additions & 1 deletion
Conversation (1)
Conversation
(
1
)
Commits (5)
Commits
(
5
)
Checks (3)
Checks
(
3
)
Files changed (4)
Files changed
(
4
)
Open
JY-20915 add alias for EU #12105 LakyLak wants to merge 5 commits into master from JY-20915-add-domain-specific-email-text-relay Copy head branch name to clipboard
JY-20915 add alias for EU
JY-20915 add alias for EU
#
12105
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Conversation
Conversation
@LakyLak
Show options
LakyLak commented yesterday
LakyLak
LakyLak
commented
yesterday
yesterday
JIRA: JY-20915
JIRA:
JY-20915
JY-20915
Changes:
Changes:
Support text relay for EU
Prevent errors from EU on US
Add or remove reactions...
|
[{"role":"AXRadioButton","text [{"role":"AXRadioButton","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":4,"bounds":{"left":0.0,"top":0.0518755,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.06304868,"width":0.10106383,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":4,"bounds":{"left":0.0,"top":0.08459697,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.09577015,"width":0.10721409,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.11731844,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.12849163,"width":0.15791224,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.15003991,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.16121309,"width":0.1796875,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.18276137,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.19393456,"width":0.12699468,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Close tab","depth":5,"bounds":{"left":0.06732048,"top":0.18994413,"width":0.007978723,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXRadioButton","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":4,"bounds":{"left":0.0,"top":0.21548285,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":5,"bounds":{"left":0.013297873,"top":0.22665602,"width":0.10139628,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":4,"bounds":{"left":0.0,"top":0.2482043,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.25937748,"width":0.18068483,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":4,"bounds":{"left":0.0,"top":0.28092578,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":5,"bounds":{"left":0.013297873,"top":0.29209897,"width":0.4644282,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Pipelines - jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.31364724,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Pipelines - jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.32482043,"width":0.039228722,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.3463687,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.3575419,"width":0.15824468,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Tab","depth":4,"bounds":{"left":0.0028257978,"top":0.38068634,"width":0.07413564,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Customize sidebar","depth":6,"bounds":{"left":0.0028257978,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Close Google Gemini (⌃X)","depth":6,"bounds":{"left":0.013796543,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Tabs from other devices","depth":6,"bounds":{"left":0.024933511,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open history (⇧⌘H)","depth":6,"bounds":{"left":0.036070477,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open bookmarks (⌘B)","depth":6,"bounds":{"left":0.04720745,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"AI Chat settings","depth":7,"bounds":{"left":0.35854387,"top":0.055067837,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Close","depth":7,"bounds":{"left":0.37051198,"top":0.055067837,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Main menu","depth":13,"bounds":{"left":0.08228058,"top":0.09736632,"width":0.015957447,"height":0.03830806},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"New chat","depth":12,"bounds":{"left":0.09823803,"top":0.10215483,"width":0.028091755,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Gemini","depth":15,"bounds":{"left":0.1008976,"top":0.10654429,"width":0.02144282,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Chat","depth":13,"bounds":{"left":0.35854387,"top":0.10215483,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Open menu for conversation actions.","depth":13,"bounds":{"left":0.37051198,"top":0.10215483,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Conversation with Gemini","depth":15,"bounds":{"left":0.079288565,"top":0.14445332,"width":0.0003324468,"height":0.0007980846},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Conversation with Gemini","depth":16,"bounds":{"left":0.079288565,"top":0.14684756,"width":0.1200133,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"You said I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.","depth":23,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Expand","depth":21,"on_screen":false,"role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a concise summary of the changes made in the code selection:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Refactoring and Dependencies","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Refactoring and Dependencies","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Replaced Interface with Implementation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Changed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusInterface","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusManager","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"in the imports and constructor dependency injection.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Redis Support:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Imported","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Illuminate\\Support\\Facades\\Redis","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to handle a newly introduced memory-logging lock mechanism.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Traits:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"WorkerIdTrait","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", along with the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"worker","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"field tracking in the final logs.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Loop and Worker Lifecycle Adjustments","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Loop and Worker Lifecycle Adjustments","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Worker Auto-Restart:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Deleted the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property, the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"constant, and the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"shouldIterateAgain()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method. The script no longer tracks execution iterations or triggers a self-managed worker restart.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Simplified Chunk Method Return:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Modified","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to always return","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Priority Handling for Throttled Entities","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Priority Handling for Throttled Entities","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Stale Record Elevation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Updated the logic in","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"rescheduleThrottledEntities()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to check if a throttled entity group is classified as \"stale\". If it is, the entities are rescheduled using","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"QueuePriorityEnum::NORMAL","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"; otherwise, they keep their respective queue priority.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Metrics, Logging, and Environment Handling","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Metrics, Logging, and Environment Handling","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Environment Tracking:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Introduced a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$environment","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property (defaulting to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"'local'","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") and a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"setEnvironment()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method to dynamically assign it.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Memory Monitoring via Datadog Gauge:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method that submits peak memory allocation to Datadog as a gauge metric (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"jiminny.reindex-memory-usage","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis Lock for Metrics Throttling:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Implemented a Redis-backed lock (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"%s-process-manager-lock","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") with a 60-second expiration inside","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to prevent over-reporting metrics.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Log Payload Update:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added total processed items (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$updatedDocuments + $deletedDocuments","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") to the final information log.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":23,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"You said are there any issues or potential bottlenecks","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"are there any issues or potential bottlenecks","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Edit","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a breakdown of the potential issues to look out for:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"1. Memory Leak Risk (OOM Crashes)","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"1. Memory Leak Risk (OOM Crashes)","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Restart Logic:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The most significant change is the removal of the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Dependency Leaks:","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.050033245,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If any underlying service (like the","depth":26,"bounds":{"left":0.17453457,"top":0.0,"width":0.08277926,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ElasticSearchClient","depth":27,"bounds":{"left":0.25930852,"top":0.0,"width":0.056848403,"height":0.015163607},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", ORM, or Datadog helper) retains references to processed entities,","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.21775267,"height":0.03631285},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"bounds":{"left":0.26811835,"top":0.0,"width":0.056848403,"height":0.015163607},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.23138298,"height":0.05546688},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Hardcoded Limits:","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.047041222,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The code still includes","depth":26,"bounds":{"left":0.17154256,"top":0.0,"width":0.05651596,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ini_set('memory_limit', '512M');","depth":27,"bounds":{"left":0.23005319,"top":0.0,"width":0.095744684,"height":0.015163607},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":". Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.23055187,"height":0.05546688},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"2. Race Condition in Redis Lock","depth":23,"bounds":{"left":0.111369684,"top":0.07940942,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"2. Race Condition in Redis Lock","depth":24,"bounds":{"left":0.111369684,"top":0.079010375,"width":0.09125665,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Non-Atomic Operations:","depth":26,"bounds":{"left":0.12450133,"top":0.10614525,"width":0.062333778,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The","depth":26,"bounds":{"left":0.18683511,"top":0.10614525,"width":0.011635638,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage","depth":27,"bounds":{"left":0.20046543,"top":0.10774142,"width":0.041888297,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method uses three distinct steps to handle the lock:","depth":26,"bounds":{"left":0.12450133,"top":0.10614525,"width":0.22722739,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"get()","depth":27,"bounds":{"left":0.14860372,"top":0.12689546,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":",","depth":26,"bounds":{"left":0.16555852,"top":0.12529927,"width":0.002493351,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"bounds":{"left":0.17004654,"top":0.12689546,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", and","depth":26,"bounds":{"left":0.18700133,"top":0.12529927,"width":0.013131649,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"expire()","depth":27,"bounds":{"left":0.20212767,"top":0.12689546,"width":0.023936171,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"bounds":{"left":0.22805852,"top":0.12529927,"width":0.0013297872,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"bounds":{"left":0.12450133,"top":0.15403032,"width":0.040059842,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If multiple workers process the same","depth":26,"bounds":{"left":0.16456117,"top":0.15403032,"width":0.09275266,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$this->updateTarget","depth":27,"bounds":{"left":0.25930852,"top":0.15562649,"width":0.056848403,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"concurrently, they can both pass the","depth":26,"bounds":{"left":0.12450133,"top":0.15403032,"width":0.22855718,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"if (! is_null(Redis::get(...)))","depth":27,"bounds":{"left":0.18351063,"top":0.17478053,"width":0.09275266,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"check at the exact same millisecond. Both will then execute","depth":26,"bounds":{"left":0.12450133,"top":0.17318435,"width":0.21575798,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"bounds":{"left":0.2122673,"top":0.19393456,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and push duplicate metrics to Datadog.","depth":26,"bounds":{"left":0.22922207,"top":0.19233839,"width":0.09823803,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Fix:","depth":26,"bounds":{"left":0.12450133,"top":0.22106944,"width":0.019281914,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"This should be replaced with a single atomic operation using Redis's","depth":26,"bounds":{"left":0.14378324,"top":0.22106944,"width":0.16821809,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"NX","depth":27,"bounds":{"left":0.31399602,"top":0.22266561,"width":0.005984043,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Not eXists) and","depth":26,"bounds":{"left":0.12450133,"top":0.22106944,"width":0.22855718,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"EX","depth":27,"bounds":{"left":0.13696809,"top":0.24181964,"width":0.005984043,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Expire) flags, or Laravel’s built-in atomic cache locks:","depth":26,"bounds":{"left":0.14494681,"top":0.24022347,"width":0.13264628,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis::set($gaugeLockName, true, 'EX', 60, 'NX')","depth":27,"bounds":{"left":0.12649602,"top":0.26097366,"width":0.14361702,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"bounds":{"left":0.27210772,"top":0.25937748,"width":0.0013297872,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"3. Infinite Loops and Caller Contract","depth":23,"bounds":{"left":0.111369684,"top":0.30925778,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"3. Infinite Loops and Caller Contract","depth":24,"bounds":{"left":0.111369684,"top":0.30885875,"width":0.10571808,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Return Value Change:","depth":26,"bounds":{"left":0.12450133,"top":0.33599362,"width":0.055352394,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"bounds":{"left":0.18301196,"top":0.33758977,"width":0.06881649,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"previously returned whether it should iterate again based on staleness or chunk size. It now always returns","depth":26,"bounds":{"left":0.12450133,"top":0.33599362,"width":0.2237367,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"bounds":{"left":0.29438165,"top":0.3567438,"width":0.011968086,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(unless the selection is totally empty).","depth":26,"bounds":{"left":0.12450133,"top":0.35514766,"width":0.2130984,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"bounds":{"left":0.12450133,"top":0.40303272,"width":0.040059842,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the code calling this method relies on a","depth":26,"bounds":{"left":0.16456117,"top":0.40303272,"width":0.10305851,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"false","depth":27,"bounds":{"left":0.26961437,"top":0.4046289,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"return to pause, sleep, or gracefully terminate, changing this to an unconditional","depth":26,"bounds":{"left":0.12450133,"top":0.40303272,"width":0.22589761,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"bounds":{"left":0.26163563,"top":0.4237829,"width":0.011968086,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"could result in an aggressive \"busy loop\" that spikes CPU usage.","depth":26,"bounds":{"left":0.12450133,"top":0.42218676,"width":0.22290559,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"4. Performance Overhead","depth":23,"bounds":{"left":0.111369684,"top":0.49122107,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"4. Performance Overhead","depth":24,"bounds":{"left":0.111369684,"top":0.49082202,"width":0.076130316,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Expensive Garbage Collection:","depth":26,"bounds":{"left":0.12450133,"top":0.5179569,"width":0.078457445,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Calling","depth":26,"bounds":{"left":0.20295878,"top":0.5179569,"width":0.019281914,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles();","depth":27,"bounds":{"left":0.22423537,"top":0.51955307,"width":0.059840426,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"on","depth":26,"bounds":{"left":0.28607047,"top":0.5179569,"width":0.00880984,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"every single chunk","depth":26,"bounds":{"left":0.29488033,"top":0.5179569,"width":0.045545213,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.","depth":26,"bounds":{"left":0.12450133,"top":0.5179569,"width":0.23420878,"height":0.05546688},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Synchronous Metric Dispatch:","depth":26,"bounds":{"left":0.12450133,"top":0.584996,"width":0.07712766,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the","depth":26,"bounds":{"left":0.20162898,"top":0.584996,"width":0.015292553,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::increment","depth":27,"bounds":{"left":0.21891622,"top":0.5865922,"width":0.053856384,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"or","depth":26,"bounds":{"left":0.27476728,"top":0.584996,"width":0.0078125,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::gauge","depth":27,"bounds":{"left":0.28457448,"top":0.5865922,"width":0.041888297,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.","depth":26,"bounds":{"left":0.12450133,"top":0.584996,"width":0.23420878,"height":0.05546688},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"5. Lack of Explicit Error Handling","depth":23,"bounds":{"left":0.111369684,"top":0.67318434,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5. Lack of Explicit Error Handling","depth":24,"bounds":{"left":0.111369684,"top":0.67278534,"width":0.094913565,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"No","depth":26,"bounds":{"left":0.12450133,"top":0.6999202,"width":0.008477394,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"try/catch","depth":27,"bounds":{"left":0.1349734,"top":0.7015164,"width":0.026928192,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Blocks:","depth":26,"bounds":{"left":0.16389628,"top":0.6999202,"width":0.019614361,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The script executes complex network operations (","depth":26,"bounds":{"left":0.18351063,"top":0.6999202,"width":0.12367021,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"loadDocuments","depth":27,"bounds":{"left":0.30917552,"top":0.7015164,"width":0.038896278,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.","depth":26,"bounds":{"left":0.12450133,"top":0.6999202,"width":0.22888963,"height":0.09377494},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":22,"bounds":{"left":0.1087101,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":22,"bounds":{"left":0.124667555,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Redo","depth":22,"bounds":{"left":0.140625,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":21,"bounds":{"left":0.15658244,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":22,"bounds":{"left":0.17253989,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":21,"bounds":{"left":0.18849733,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXTextArea","text":"Ask Gemini","depth":21,"bounds":{"left":0.14594415,"top":0.86632085,"width":0.17021276,"height":0.01915403},"on_screen":true,"value":"Ask Gemini","help_text":"","role_description":"text entry area","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Ask Gemini","depth":22,"bounds":{"left":0.14594415,"top":0.8667199,"width":0.030418882,"height":0.018355945},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Upload & tools","depth":20,"bounds":{"left":0.12732713,"top":0.8567438,"width":0.015957447,"height":0.03830806},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Microphone","depth":20,"bounds":{"left":0.32413563,"top":0.86153233,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Send message","depth":20,"bounds":{"left":0.33211437,"top":0.8631285,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":false,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.","depth":17,"bounds":{"left":0.102726065,"top":0.9173983,"width":0.21775267,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Your privacy & Gemini Opens in a new window","depth":17,"bounds":{"left":0.32047874,"top":0.9173983,"width":0.043882977,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Your privacy & Gemini","depth":18,"bounds":{"left":0.32047874,"top":0.9173983,"width":0.043882977,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Opens in a new window","depth":19,"bounds":{"left":0.079288565,"top":0.91660017,"width":0.04720745,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Summarize page","depth":7,"bounds":{"left":0.08494016,"top":0.95730245,"width":0.053523935,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Summarize page","depth":9,"bounds":{"left":0.09059176,"top":0.96249,"width":0.042220745,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Skip to content","depth":6,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Skip to content","depth":7,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Open menu","depth":10,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"Homepage (g then d)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"jiminny","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"jiminny","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"app","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"app","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Search or jump to…","depth":9,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Type","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"/","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to search","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Chat with Copilot","depth":10,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXMenuButton","text":"Open Copilot…","depth":9,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXMenuButton","text":"Create new...","depth":9,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"All issues(g then i)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"All pull requests","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"All repositories","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"You have unread notifications(g then n)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Open user navigation menu","depth":9,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Repository navigation","depth":9,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Repository navigation","depth":10,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Code","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Code","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Pull requests (32)","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Pull requests","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"32","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Agents","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Agents","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Actions","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Actions","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Wiki","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Wiki","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Security and quality","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Security and quality","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Insights","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Insights","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Settings","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Settings","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"JY-20915 add alias for EU #12105 Edit title","depth":13,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"JY-20915 add alias for EU","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"#","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"12105","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Edit title","depth":14,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Awaiting approval","depth":13,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Awaiting approval","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXMenuButton","text":"Code","depth":13,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXStaticText","text":"Code","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Open","depth":13,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"LakyLak","depth":15,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"wants to merge 5 commits into","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"master","depth":15,"on_screen":false,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"master","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"from","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915-add-domain-specific-email-text-relay","depth":16,"on_screen":false,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915-add-domain-specific-email-text-relay","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy head branch name to clipboard","depth":16,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Lines changed: 365 additions & 1 deletion","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Conversation (1)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"Conversation","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"1","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Commits (5)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Commits","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Checks (3)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Checks","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"3","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Files changed (4)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Files changed","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"4","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Open","depth":14,"bounds":{"left":0.50332445,"top":0.07222666,"width":0.011968086,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"JY-20915 add alias for EU #12105 LakyLak wants to merge 5 commits into master from JY-20915-add-domain-specific-email-text-relay Copy head branch name to clipboard","depth":14,"bounds":{"left":0.5219415,"top":0.058260176,"width":0.22922207,"height":0.042298485},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915 add alias for EU","depth":16,"bounds":{"left":0.5219415,"top":0.05865922,"width":0.05817819,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915 add alias for EU","depth":17,"bounds":{"left":0.5219415,"top":0.06304868,"width":0.05817819,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"#","depth":16,"bounds":{"left":0.5827792,"top":0.06304868,"width":0.0028257978,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"12105","depth":16,"bounds":{"left":0.585605,"top":0.06304868,"width":0.012799202,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"LakyLak","depth":18,"bounds":{"left":0.5219415,"top":0.08339984,"width":0.016123671,"height":0.011971269},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":19,"bounds":{"left":0.5219415,"top":0.08339984,"width":0.016123671,"height":0.011971269},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"wants to merge 5 commits into","depth":18,"bounds":{"left":0.539395,"top":0.08339984,"width":0.058011968,"height":0.011971269},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"master","depth":18,"bounds":{"left":0.5987367,"top":0.08180367,"width":0.018450798,"height":0.015163607},"on_screen":true,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"master","depth":19,"bounds":{"left":0.6007314,"top":0.08339984,"width":0.014461436,"height":0.011572227},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"from","depth":19,"bounds":{"left":0.6185173,"top":0.08339984,"width":0.008643617,"height":0.011971269},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915-add-domain-specific-email-text-relay","depth":19,"bounds":{"left":0.6284907,"top":0.08180367,"width":0.112034574,"height":0.015163607},"on_screen":true,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915-add-domain-specific-email-text-relay","depth":20,"bounds":{"left":0.63048536,"top":0.08339984,"width":0.10804521,"height":0.011572227},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy head branch name to clipboard","depth":19,"bounds":{"left":0.741855,"top":0.07821229,"width":0.00930851,"height":0.022346368},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Conversation","depth":12,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Conversation","depth":13,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@LakyLak","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show options","depth":15,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXHeading","text":"LakyLak commented yesterday","depth":14,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXLink","text":"LakyLak","depth":16,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"commented","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"yesterday","depth":15,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"yesterday","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"JIRA: JY-20915","depth":16,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"JIRA:","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915","depth":17,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Changes:","depth":16,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Changes:","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Support text relay for EU","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Prevent errors from EU on US","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Add or remove reactions","depth":16,"on_screen":false,"help_text":"","role_description":"summary","subrole":"AXSummary","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false}]...
|
672243981171529941
|
-1048292410045221092
|
idle
|
accessibility
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo
Share & export
Copy
Show more options
Ask Gemini
Ask Gemini
Upload & tools
Microphone
Send message
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Summarize page
Summarize page
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
All issues(g then i)
All pull requests
All repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Pull requests (32)
Pull requests
(
32
)
Agents
Agents
Actions
Actions
Wiki
Wiki
Security and quality
Security and quality
Insights
Insights
Settings
Settings
JY-20915 add alias for EU #12105 Edit title
JY-20915 add alias for EU
#
12105
Edit title
Awaiting approval
Awaiting approval
Code
Code
Open
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Lines changed: 365 additions & 1 deletion
Conversation (1)
Conversation
(
1
)
Commits (5)
Commits
(
5
)
Checks (3)
Checks
(
3
)
Files changed (4)
Files changed
(
4
)
Open
JY-20915 add alias for EU #12105 LakyLak wants to merge 5 commits into master from JY-20915-add-domain-specific-email-text-relay Copy head branch name to clipboard
JY-20915 add alias for EU
JY-20915 add alias for EU
#
12105
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Conversation
Conversation
@LakyLak
Show options
LakyLak commented yesterday
LakyLak
LakyLak
commented
yesterday
yesterday
JIRA: JY-20915
JIRA:
JY-20915
JY-20915
Changes:
Changes:
Support text relay for EU
Prevent errors from EU on US
Add or remove reactions...
|
65246
|
NULL
|
NULL
|
NULL
|
|
65249
|
2287
|
8
|
2026-05-20T15:34:19.770038+00:00
|
/Users/lukas/.screenpipe/data/data/2026-05-20/1779 /Users/lukas/.screenpipe/data/data/2026-05-20/1779291259770_m1.jpg...
|
Firefox
|
JY-20915 add alias for EU by LakyLak · Pull Reques JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app — Work...
|
1
|
github.com/jiminny/app/pull/12105
|
monitor_1
|
NULL
|
NULL
|
NULL
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo
Share & export
Copy
Show more options
Ask Gemini
Ask Gemini
Upload & tools
Microphone
Send message
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Summarize page
Summarize page
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
All issues(g then i)
All pull requests
All repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code...
|
[{"role":"AXRadioButton","text [{"role":"AXRadioButton","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Close tab","depth":5,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXRadioButton","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Pipelines - jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Pipelines - jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Tab","depth":4,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Customize sidebar","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Close Google Gemini (⌃X)","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Tabs from other devices","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open history (⇧⌘H)","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open bookmarks (⌘B)","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"AI Chat settings","depth":7,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Close","depth":7,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Main menu","depth":13,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"New chat","depth":12,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Gemini","depth":15,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Chat","depth":13,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Open menu for conversation actions.","depth":13,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Conversation with Gemini","depth":15,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Conversation with Gemini","depth":16,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"You said I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.","depth":23,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Expand","depth":21,"on_screen":false,"role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a concise summary of the changes made in the code selection:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Refactoring and Dependencies","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Refactoring and Dependencies","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Replaced Interface with Implementation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Changed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusInterface","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusManager","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"in the imports and constructor dependency injection.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Redis Support:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Imported","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Illuminate\\Support\\Facades\\Redis","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to handle a newly introduced memory-logging lock mechanism.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Traits:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"WorkerIdTrait","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", along with the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"worker","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"field tracking in the final logs.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Loop and Worker Lifecycle Adjustments","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Loop and Worker Lifecycle Adjustments","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Worker Auto-Restart:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Deleted the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property, the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"constant, and the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"shouldIterateAgain()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method. The script no longer tracks execution iterations or triggers a self-managed worker restart.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Simplified Chunk Method Return:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Modified","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to always return","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Priority Handling for Throttled Entities","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Priority Handling for Throttled Entities","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Stale Record Elevation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Updated the logic in","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"rescheduleThrottledEntities()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to check if a throttled entity group is classified as \"stale\". If it is, the entities are rescheduled using","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"QueuePriorityEnum::NORMAL","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"; otherwise, they keep their respective queue priority.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Metrics, Logging, and Environment Handling","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Metrics, Logging, and Environment Handling","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Environment Tracking:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Introduced a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$environment","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property (defaulting to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"'local'","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") and a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"setEnvironment()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method to dynamically assign it.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Memory Monitoring via Datadog Gauge:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method that submits peak memory allocation to Datadog as a gauge metric (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"jiminny.reindex-memory-usage","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis Lock for Metrics Throttling:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Implemented a Redis-backed lock (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"%s-process-manager-lock","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") with a 60-second expiration inside","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to prevent over-reporting metrics.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Log Payload Update:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added total processed items (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$updatedDocuments + $deletedDocuments","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") to the final information log.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":23,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"You said are there any issues or potential bottlenecks","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"are there any issues or potential bottlenecks","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Edit","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a breakdown of the potential issues to look out for:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"1. Memory Leak Risk (OOM Crashes)","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"1. Memory Leak Risk (OOM Crashes)","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Restart Logic:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The most significant change is the removal of the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Dependency Leaks:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If any underlying service (like the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ElasticSearchClient","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", ORM, or Datadog helper) retains references to processed entities,","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Hardcoded Limits:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The code still includes","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ini_set('memory_limit', '512M');","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":". Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"2. Race Condition in Redis Lock","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"2. Race Condition in Redis Lock","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Non-Atomic Operations:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method uses three distinct steps to handle the lock:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"get()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":",","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", and","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"expire()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If multiple workers process the same","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$this->updateTarget","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"concurrently, they can both pass the","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"if (! is_null(Redis::get(...)))","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"check at the exact same millisecond. Both will then execute","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and push duplicate metrics to Datadog.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Fix:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"This should be replaced with a single atomic operation using Redis's","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"NX","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Not eXists) and","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"EX","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Expire) flags, or Laravel’s built-in atomic cache locks:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis::set($gaugeLockName, true, 'EX', 60, 'NX')","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"3. Infinite Loops and Caller Contract","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"3. Infinite Loops and Caller Contract","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Return Value Change:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"previously returned whether it should iterate again based on staleness or chunk size. It now always returns","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(unless the selection is totally empty).","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the code calling this method relies on a","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"false","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"return to pause, sleep, or gracefully terminate, changing this to an unconditional","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"could result in an aggressive \"busy loop\" that spikes CPU usage.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"4. Performance Overhead","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"4. Performance Overhead","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Expensive Garbage Collection:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Calling","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles();","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"on","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"every single chunk","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Synchronous Metric Dispatch:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::increment","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"or","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::gauge","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"5. Lack of Explicit Error Handling","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5. Lack of Explicit Error Handling","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"No","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"try/catch","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Blocks:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The script executes complex network operations (","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"loadDocuments","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":22,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":22,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Redo","depth":22,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":21,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":22,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":21,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXTextArea","text":"Ask Gemini","depth":21,"on_screen":true,"value":"Ask Gemini","help_text":"","role_description":"text entry area","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Ask Gemini","depth":22,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Upload & tools","depth":20,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Microphone","depth":20,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Send message","depth":20,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":false,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.","depth":17,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Your privacy & Gemini Opens in a new window","depth":17,"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Your privacy & Gemini","depth":18,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Opens in a new window","depth":19,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Summarize page","depth":7,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Summarize page","depth":9,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Skip to content","depth":6,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Skip to content","depth":7,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Open menu","depth":10,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"Homepage (g then d)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"jiminny","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"jiminny","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"app","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"app","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Search or jump to…","depth":9,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Type","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"/","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to search","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Chat with Copilot","depth":10,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXMenuButton","text":"Open Copilot…","depth":9,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXMenuButton","text":"Create new...","depth":9,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"All issues(g then i)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"All pull requests","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"All repositories","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"You have unread notifications(g then n)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Open user navigation menu","depth":9,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Repository navigation","depth":9,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Repository navigation","depth":10,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Code","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false}]...
|
6952831365328364286
|
-1052936747159816292
|
idle
|
accessibility
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo
Share & export
Copy
Show more options
Ask Gemini
Ask Gemini
Upload & tools
Microphone
Send message
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Summarize page
Summarize page
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
All issues(g then i)
All pull requests
All repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code...
|
65247
|
NULL
|
NULL
|
NULL
|
|
65250
|
NULL
|
0
|
2026-05-20T15:34:31.650221+00:00
|
/Users/lukas/.screenpipe/data/data/2026-05-20/1779 /Users/lukas/.screenpipe/data/data/2026-05-20/1779291271650_m2.jpg...
|
Firefox
|
JY-20915 add alias for EU by LakyLak · Pull Reques JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app — Work...
|
1
|
github.com/jiminny/app/pull/12105
|
monitor_2
|
NULL
|
NULL
|
NULL
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo
Share & export
Copy
Show more options
Ask Gemini
Ask Gemini
Upload & tools
Microphone
Send message
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Summarize page
Summarize page
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
All issues(g then i)
All pull requests
All repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Pull requests (32)
Pull requests
(
32
)
Agents
Agents
Actions
Actions
Wiki
Wiki
Security and quality
Security and quality
Insights
Insights
Settings
Settings
JY-20915 add alias for EU #12105 Edit title
JY-20915 add alias for EU
#
12105
Edit title
Awaiting approval
Awaiting approval
Code
Code
Open
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Lines changed: 365 additions & 1 deletion
Conversation (1)
Conversation
(
1
)
Commits (5)
Commits
(
5
)
Checks (3)
Checks
(
3
)
Files changed (4)
Files changed
(
4
)
Open
JY-20915 add alias for EU #12105 LakyLak wants to merge 5 commits into master from JY-20915-add-domain-specific-email-text-relay Copy head branch name to clipboard
JY-20915 add alias for EU
JY-20915 add alias for EU
#
12105
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Conversation
Conversation
@LakyLak
Show options
LakyLak commented yesterday
LakyLak
LakyLak
commented
yesterday
yesterday
JIRA: JY-20915
JIRA:
JY-20915
JY-20915
Changes:
Changes:
Support text relay for EU
Prevent errors from EU on US
Add or remove reactions
LakyLak
LakyLak
added
5
commits
yesterday
yesterday
@LakyLak
JY-20915
JY-20915
add alias for EU
add alias for EU
8 / 10 checks OK
e5255d5
e5255d5
@LakyLak
JY-20915
JY-20915
fix env recognization
fix env recognization
11 / 12 checks OK
8953054
8953054
@LakyLak
JY-20915
JY-20915
code smells & tests
code smells & tests
11 / 12 checks OK
73d4bea
73d4bea
@LakyLak
JY-20915
JY-20915
add more tests
add more tests
11 / 12 checks OK
5d2dd3e
5d2dd3e
@LakyLak
JY-20915
JY-20915
add sms received test...
|
[{"role":"AXRadioButton","text [{"role":"AXRadioButton","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":4,"bounds":{"left":0.0,"top":0.0518755,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.06304868,"width":0.10106383,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":4,"bounds":{"left":0.0,"top":0.08459697,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.09577015,"width":0.10721409,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.11731844,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.12849163,"width":0.15791224,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.15003991,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.16121309,"width":0.1796875,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.18276137,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.19393456,"width":0.12699468,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Close tab","depth":5,"bounds":{"left":0.06732048,"top":0.18994413,"width":0.007978723,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXRadioButton","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":4,"bounds":{"left":0.0,"top":0.21548285,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":5,"bounds":{"left":0.013297873,"top":0.22665602,"width":0.10139628,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":4,"bounds":{"left":0.0,"top":0.2482043,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.25937748,"width":0.18068483,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":4,"bounds":{"left":0.0,"top":0.28092578,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":5,"bounds":{"left":0.013297873,"top":0.29209897,"width":0.4644282,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Pipelines - jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.31364724,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Pipelines - jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.32482043,"width":0.039228722,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.3463687,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.3575419,"width":0.15824468,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Tab","depth":4,"bounds":{"left":0.0028257978,"top":0.38068634,"width":0.07413564,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Customize sidebar","depth":6,"bounds":{"left":0.0028257978,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Close Google Gemini (⌃X)","depth":6,"bounds":{"left":0.013796543,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Tabs from other devices","depth":6,"bounds":{"left":0.024933511,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open history (⇧⌘H)","depth":6,"bounds":{"left":0.036070477,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open bookmarks (⌘B)","depth":6,"bounds":{"left":0.04720745,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"AI Chat settings","depth":7,"bounds":{"left":0.35854387,"top":0.055067837,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Close","depth":7,"bounds":{"left":0.37051198,"top":0.055067837,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Main menu","depth":13,"bounds":{"left":0.08228058,"top":0.09736632,"width":0.015957447,"height":0.03830806},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"New chat","depth":12,"bounds":{"left":0.09823803,"top":0.10215483,"width":0.028091755,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Gemini","depth":15,"bounds":{"left":0.1008976,"top":0.10654429,"width":0.02144282,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Chat","depth":13,"bounds":{"left":0.35854387,"top":0.10215483,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Open menu for conversation actions.","depth":13,"bounds":{"left":0.37051198,"top":0.10215483,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Conversation with Gemini","depth":15,"bounds":{"left":0.079288565,"top":0.14445332,"width":0.0003324468,"height":0.0007980846},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Conversation with Gemini","depth":16,"bounds":{"left":0.079288565,"top":0.14684756,"width":0.1200133,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"You said I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.","depth":23,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Expand","depth":21,"on_screen":false,"role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a concise summary of the changes made in the code selection:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Refactoring and Dependencies","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Refactoring and Dependencies","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Replaced Interface with Implementation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Changed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusInterface","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusManager","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"in the imports and constructor dependency injection.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Redis Support:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Imported","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Illuminate\\Support\\Facades\\Redis","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to handle a newly introduced memory-logging lock mechanism.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Traits:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"WorkerIdTrait","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", along with the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"worker","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"field tracking in the final logs.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Loop and Worker Lifecycle Adjustments","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Loop and Worker Lifecycle Adjustments","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Worker Auto-Restart:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Deleted the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property, the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"constant, and the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"shouldIterateAgain()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method. The script no longer tracks execution iterations or triggers a self-managed worker restart.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Simplified Chunk Method Return:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Modified","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to always return","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Priority Handling for Throttled Entities","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Priority Handling for Throttled Entities","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Stale Record Elevation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Updated the logic in","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"rescheduleThrottledEntities()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to check if a throttled entity group is classified as \"stale\". If it is, the entities are rescheduled using","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"QueuePriorityEnum::NORMAL","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"; otherwise, they keep their respective queue priority.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Metrics, Logging, and Environment Handling","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Metrics, Logging, and Environment Handling","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Environment Tracking:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Introduced a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$environment","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property (defaulting to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"'local'","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") and a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"setEnvironment()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method to dynamically assign it.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Memory Monitoring via Datadog Gauge:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method that submits peak memory allocation to Datadog as a gauge metric (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"jiminny.reindex-memory-usage","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis Lock for Metrics Throttling:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Implemented a Redis-backed lock (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"%s-process-manager-lock","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") with a 60-second expiration inside","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to prevent over-reporting metrics.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Log Payload Update:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added total processed items (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$updatedDocuments + $deletedDocuments","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") to the final information log.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":23,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"You said are there any issues or potential bottlenecks","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"are there any issues or potential bottlenecks","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Edit","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a breakdown of the potential issues to look out for:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"1. Memory Leak Risk (OOM Crashes)","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"1. Memory Leak Risk (OOM Crashes)","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Restart Logic:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The most significant change is the removal of the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Dependency Leaks:","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.050033245,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If any underlying service (like the","depth":26,"bounds":{"left":0.17453457,"top":0.0,"width":0.08277926,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ElasticSearchClient","depth":27,"bounds":{"left":0.25930852,"top":0.0,"width":0.056848403,"height":0.015163607},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", ORM, or Datadog helper) retains references to processed entities,","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.21775267,"height":0.03631285},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"bounds":{"left":0.26811835,"top":0.0,"width":0.056848403,"height":0.015163607},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.23138298,"height":0.05546688},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Hardcoded Limits:","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.047041222,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The code still includes","depth":26,"bounds":{"left":0.17154256,"top":0.0,"width":0.05651596,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ini_set('memory_limit', '512M');","depth":27,"bounds":{"left":0.23005319,"top":0.0,"width":0.095744684,"height":0.015163607},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":". Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.23055187,"height":0.05546688},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"2. Race Condition in Redis Lock","depth":23,"bounds":{"left":0.111369684,"top":0.07940942,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"2. Race Condition in Redis Lock","depth":24,"bounds":{"left":0.111369684,"top":0.079010375,"width":0.09125665,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Non-Atomic Operations:","depth":26,"bounds":{"left":0.12450133,"top":0.10614525,"width":0.062333778,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The","depth":26,"bounds":{"left":0.18683511,"top":0.10614525,"width":0.011635638,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage","depth":27,"bounds":{"left":0.20046543,"top":0.10774142,"width":0.041888297,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method uses three distinct steps to handle the lock:","depth":26,"bounds":{"left":0.12450133,"top":0.10614525,"width":0.22722739,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"get()","depth":27,"bounds":{"left":0.14860372,"top":0.12689546,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":",","depth":26,"bounds":{"left":0.16555852,"top":0.12529927,"width":0.002493351,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"bounds":{"left":0.17004654,"top":0.12689546,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", and","depth":26,"bounds":{"left":0.18700133,"top":0.12529927,"width":0.013131649,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"expire()","depth":27,"bounds":{"left":0.20212767,"top":0.12689546,"width":0.023936171,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"bounds":{"left":0.22805852,"top":0.12529927,"width":0.0013297872,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"bounds":{"left":0.12450133,"top":0.15403032,"width":0.040059842,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If multiple workers process the same","depth":26,"bounds":{"left":0.16456117,"top":0.15403032,"width":0.09275266,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$this->updateTarget","depth":27,"bounds":{"left":0.25930852,"top":0.15562649,"width":0.056848403,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"concurrently, they can both pass the","depth":26,"bounds":{"left":0.12450133,"top":0.15403032,"width":0.22855718,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"if (! is_null(Redis::get(...)))","depth":27,"bounds":{"left":0.18351063,"top":0.17478053,"width":0.09275266,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"check at the exact same millisecond. Both will then execute","depth":26,"bounds":{"left":0.12450133,"top":0.17318435,"width":0.21575798,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"bounds":{"left":0.2122673,"top":0.19393456,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and push duplicate metrics to Datadog.","depth":26,"bounds":{"left":0.22922207,"top":0.19233839,"width":0.09823803,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Fix:","depth":26,"bounds":{"left":0.12450133,"top":0.22106944,"width":0.019281914,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"This should be replaced with a single atomic operation using Redis's","depth":26,"bounds":{"left":0.14378324,"top":0.22106944,"width":0.16821809,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"NX","depth":27,"bounds":{"left":0.31399602,"top":0.22266561,"width":0.005984043,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Not eXists) and","depth":26,"bounds":{"left":0.12450133,"top":0.22106944,"width":0.22855718,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"EX","depth":27,"bounds":{"left":0.13696809,"top":0.24181964,"width":0.005984043,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Expire) flags, or Laravel’s built-in atomic cache locks:","depth":26,"bounds":{"left":0.14494681,"top":0.24022347,"width":0.13264628,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis::set($gaugeLockName, true, 'EX', 60, 'NX')","depth":27,"bounds":{"left":0.12649602,"top":0.26097366,"width":0.14361702,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"bounds":{"left":0.27210772,"top":0.25937748,"width":0.0013297872,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"3. Infinite Loops and Caller Contract","depth":23,"bounds":{"left":0.111369684,"top":0.30925778,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"3. Infinite Loops and Caller Contract","depth":24,"bounds":{"left":0.111369684,"top":0.30885875,"width":0.10571808,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Return Value Change:","depth":26,"bounds":{"left":0.12450133,"top":0.33599362,"width":0.055352394,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"bounds":{"left":0.18301196,"top":0.33758977,"width":0.06881649,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"previously returned whether it should iterate again based on staleness or chunk size. It now always returns","depth":26,"bounds":{"left":0.12450133,"top":0.33599362,"width":0.2237367,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"bounds":{"left":0.29438165,"top":0.3567438,"width":0.011968086,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(unless the selection is totally empty).","depth":26,"bounds":{"left":0.12450133,"top":0.35514766,"width":0.2130984,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"bounds":{"left":0.12450133,"top":0.40303272,"width":0.040059842,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the code calling this method relies on a","depth":26,"bounds":{"left":0.16456117,"top":0.40303272,"width":0.10305851,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"false","depth":27,"bounds":{"left":0.26961437,"top":0.4046289,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"return to pause, sleep, or gracefully terminate, changing this to an unconditional","depth":26,"bounds":{"left":0.12450133,"top":0.40303272,"width":0.22589761,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"bounds":{"left":0.26163563,"top":0.4237829,"width":0.011968086,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"could result in an aggressive \"busy loop\" that spikes CPU usage.","depth":26,"bounds":{"left":0.12450133,"top":0.42218676,"width":0.22290559,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"4. Performance Overhead","depth":23,"bounds":{"left":0.111369684,"top":0.49122107,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"4. Performance Overhead","depth":24,"bounds":{"left":0.111369684,"top":0.49082202,"width":0.076130316,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Expensive Garbage Collection:","depth":26,"bounds":{"left":0.12450133,"top":0.5179569,"width":0.078457445,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Calling","depth":26,"bounds":{"left":0.20295878,"top":0.5179569,"width":0.019281914,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles();","depth":27,"bounds":{"left":0.22423537,"top":0.51955307,"width":0.059840426,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"on","depth":26,"bounds":{"left":0.28607047,"top":0.5179569,"width":0.00880984,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"every single chunk","depth":26,"bounds":{"left":0.29488033,"top":0.5179569,"width":0.045545213,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.","depth":26,"bounds":{"left":0.12450133,"top":0.5179569,"width":0.23420878,"height":0.05546688},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Synchronous Metric Dispatch:","depth":26,"bounds":{"left":0.12450133,"top":0.584996,"width":0.07712766,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the","depth":26,"bounds":{"left":0.20162898,"top":0.584996,"width":0.015292553,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::increment","depth":27,"bounds":{"left":0.21891622,"top":0.5865922,"width":0.053856384,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"or","depth":26,"bounds":{"left":0.27476728,"top":0.584996,"width":0.0078125,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::gauge","depth":27,"bounds":{"left":0.28457448,"top":0.5865922,"width":0.041888297,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.","depth":26,"bounds":{"left":0.12450133,"top":0.584996,"width":0.23420878,"height":0.05546688},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"5. Lack of Explicit Error Handling","depth":23,"bounds":{"left":0.111369684,"top":0.67318434,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5. Lack of Explicit Error Handling","depth":24,"bounds":{"left":0.111369684,"top":0.67278534,"width":0.094913565,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"No","depth":26,"bounds":{"left":0.12450133,"top":0.6999202,"width":0.008477394,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"try/catch","depth":27,"bounds":{"left":0.1349734,"top":0.7015164,"width":0.026928192,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Blocks:","depth":26,"bounds":{"left":0.16389628,"top":0.6999202,"width":0.019614361,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The script executes complex network operations (","depth":26,"bounds":{"left":0.18351063,"top":0.6999202,"width":0.12367021,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"loadDocuments","depth":27,"bounds":{"left":0.30917552,"top":0.7015164,"width":0.038896278,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.","depth":26,"bounds":{"left":0.12450133,"top":0.6999202,"width":0.22888963,"height":0.09377494},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":22,"bounds":{"left":0.1087101,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":22,"bounds":{"left":0.124667555,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Redo","depth":22,"bounds":{"left":0.140625,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":21,"bounds":{"left":0.15658244,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":22,"bounds":{"left":0.17253989,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":21,"bounds":{"left":0.18849733,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXTextArea","text":"Ask Gemini","depth":21,"bounds":{"left":0.14594415,"top":0.86632085,"width":0.17021276,"height":0.01915403},"on_screen":true,"value":"Ask Gemini","help_text":"","role_description":"text entry area","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Ask Gemini","depth":22,"bounds":{"left":0.14594415,"top":0.8667199,"width":0.030418882,"height":0.018355945},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Upload & tools","depth":20,"bounds":{"left":0.12732713,"top":0.8567438,"width":0.015957447,"height":0.03830806},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Microphone","depth":20,"bounds":{"left":0.32413563,"top":0.86153233,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Send message","depth":20,"bounds":{"left":0.33211437,"top":0.8631285,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":false,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.","depth":17,"bounds":{"left":0.102726065,"top":0.9173983,"width":0.21775267,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Your privacy & Gemini Opens in a new window","depth":17,"bounds":{"left":0.32047874,"top":0.9173983,"width":0.043882977,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Your privacy & Gemini","depth":18,"bounds":{"left":0.32047874,"top":0.9173983,"width":0.043882977,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Opens in a new window","depth":19,"bounds":{"left":0.079288565,"top":0.91660017,"width":0.04720745,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Summarize page","depth":7,"bounds":{"left":0.08494016,"top":0.95730245,"width":0.053523935,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Summarize page","depth":9,"bounds":{"left":0.09059176,"top":0.96249,"width":0.042220745,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Skip to content","depth":6,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Skip to content","depth":7,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Open menu","depth":10,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"Homepage (g then d)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"jiminny","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"jiminny","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"app","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"app","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Search or jump to…","depth":9,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Type","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"/","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to search","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Chat with Copilot","depth":10,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXMenuButton","text":"Open Copilot…","depth":9,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXMenuButton","text":"Create new...","depth":9,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"All issues(g then i)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"All pull requests","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"All repositories","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"You have unread notifications(g then n)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Open user navigation menu","depth":9,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Repository navigation","depth":9,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Repository navigation","depth":10,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Code","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Code","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Pull requests (32)","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Pull requests","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"32","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Agents","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Agents","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Actions","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Actions","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Wiki","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Wiki","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Security and quality","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Security and quality","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Insights","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Insights","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Settings","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Settings","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"JY-20915 add alias for EU #12105 Edit title","depth":13,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"JY-20915 add alias for EU","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"#","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"12105","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Edit title","depth":14,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Awaiting approval","depth":13,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Awaiting approval","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXMenuButton","text":"Code","depth":13,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXStaticText","text":"Code","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Open","depth":13,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"LakyLak","depth":15,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"wants to merge 5 commits into","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"master","depth":15,"on_screen":false,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"master","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"from","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915-add-domain-specific-email-text-relay","depth":16,"on_screen":false,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915-add-domain-specific-email-text-relay","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy head branch name to clipboard","depth":16,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Lines changed: 365 additions & 1 deletion","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Conversation (1)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"Conversation","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"1","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Commits (5)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Commits","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Checks (3)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Checks","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"3","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Files changed (4)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Files changed","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"4","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Open","depth":14,"bounds":{"left":0.50332445,"top":0.07222666,"width":0.011968086,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"JY-20915 add alias for EU #12105 LakyLak wants to merge 5 commits into master from JY-20915-add-domain-specific-email-text-relay Copy head branch name to clipboard","depth":14,"bounds":{"left":0.5219415,"top":0.058260176,"width":0.22922207,"height":0.042298485},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915 add alias for EU","depth":16,"bounds":{"left":0.5219415,"top":0.05865922,"width":0.05817819,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915 add alias for EU","depth":17,"bounds":{"left":0.5219415,"top":0.06304868,"width":0.05817819,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"#","depth":16,"bounds":{"left":0.5827792,"top":0.06304868,"width":0.0028257978,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"12105","depth":16,"bounds":{"left":0.585605,"top":0.06304868,"width":0.012799202,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"LakyLak","depth":18,"bounds":{"left":0.5219415,"top":0.08339984,"width":0.016123671,"height":0.011971269},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":19,"bounds":{"left":0.5219415,"top":0.08339984,"width":0.016123671,"height":0.011971269},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"wants to merge 5 commits into","depth":18,"bounds":{"left":0.539395,"top":0.08339984,"width":0.058011968,"height":0.011971269},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"master","depth":18,"bounds":{"left":0.5987367,"top":0.08180367,"width":0.018450798,"height":0.015163607},"on_screen":true,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"master","depth":19,"bounds":{"left":0.6007314,"top":0.08339984,"width":0.014461436,"height":0.011572227},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"from","depth":19,"bounds":{"left":0.6185173,"top":0.08339984,"width":0.008643617,"height":0.011971269},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915-add-domain-specific-email-text-relay","depth":19,"bounds":{"left":0.6284907,"top":0.08180367,"width":0.112034574,"height":0.015163607},"on_screen":true,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915-add-domain-specific-email-text-relay","depth":20,"bounds":{"left":0.63048536,"top":0.08339984,"width":0.10804521,"height":0.011572227},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy head branch name to clipboard","depth":19,"bounds":{"left":0.741855,"top":0.07821229,"width":0.00930851,"height":0.022346368},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Conversation","depth":12,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Conversation","depth":13,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@LakyLak","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show options","depth":15,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXHeading","text":"LakyLak commented yesterday","depth":14,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXLink","text":"LakyLak","depth":16,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"commented","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"yesterday","depth":15,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"yesterday","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"JIRA: JY-20915","depth":16,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"JIRA:","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915","depth":17,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Changes:","depth":16,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Changes:","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Support text relay for EU","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Prevent errors from EU on US","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Add or remove reactions","depth":16,"on_screen":false,"help_text":"","role_description":"summary","subrole":"AXSummary","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"LakyLak","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"added","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"commits","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"yesterday","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"yesterday","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@LakyLak","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"JY-20915","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"add alias for EU","depth":14,"on_screen":false,"help_text":"JY-20915 add alias for EU","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"add alias for EU","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"8 / 10 checks OK","depth":14,"on_screen":false,"help_text":"","role_description":"summary","subrole":"AXSummary","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"e5255d5","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"e5255d5","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@LakyLak","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"JY-20915","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"fix env recognization","depth":14,"on_screen":false,"help_text":"JY-20915 fix env recognization","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"fix env recognization","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"11 / 12 checks OK","depth":14,"on_screen":false,"help_text":"","role_description":"summary","subrole":"AXSummary","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"8953054","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"8953054","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@LakyLak","depth":12,"bounds":{"left":0.5249335,"top":0.0,"width":0.0066489363,"height":0.015961692},"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"JY-20915","depth":14,"bounds":{"left":0.53357714,"top":0.0,"width":0.019115692,"height":0.011572227},"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915","depth":15,"bounds":{"left":0.53357714,"top":0.0,"width":0.019115692,"height":0.011572227},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"code smells & tests","depth":14,"bounds":{"left":0.55502,"top":0.0,"width":0.045711435,"height":0.011572227},"on_screen":false,"help_text":"JY-20915 code smells & tests","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"code smells & tests","depth":15,"bounds":{"left":0.55502,"top":0.0,"width":0.045711435,"height":0.011572227},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"11 / 12 checks OK","depth":14,"bounds":{"left":0.7563165,"top":0.0,"width":0.005319149,"height":0.016759777},"on_screen":false,"help_text":"","role_description":"summary","subrole":"AXSummary","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"73d4bea","depth":14,"bounds":{"left":0.76296544,"top":0.0,"width":0.016954787,"height":0.011572227},"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"73d4bea","depth":15,"bounds":{"left":0.76296544,"top":0.0,"width":0.016954787,"height":0.011572227},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@LakyLak","depth":12,"bounds":{"left":0.5249335,"top":0.0,"width":0.0066489363,"height":0.015961692},"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"JY-20915","depth":14,"bounds":{"left":0.53357714,"top":0.0,"width":0.019115692,"height":0.011572227},"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915","depth":15,"bounds":{"left":0.53357714,"top":0.0,"width":0.019115692,"height":0.011572227},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"add more tests","depth":14,"bounds":{"left":0.55502,"top":0.0,"width":0.03357713,"height":0.011572227},"on_screen":false,"help_text":"JY-20915 add more tests","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"add more tests","depth":15,"bounds":{"left":0.55502,"top":0.0,"width":0.03357713,"height":0.011572227},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"11 / 12 checks OK","depth":14,"bounds":{"left":0.7563165,"top":0.0,"width":0.005319149,"height":0.016759777},"on_screen":false,"help_text":"","role_description":"summary","subrole":"AXSummary","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"5d2dd3e","depth":14,"bounds":{"left":0.76296544,"top":0.0,"width":0.016954787,"height":0.011572227},"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"5d2dd3e","depth":15,"bounds":{"left":0.76296544,"top":0.0,"width":0.016954787,"height":0.011572227},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@LakyLak","depth":12,"bounds":{"left":0.5249335,"top":0.0,"width":0.0066489363,"height":0.015961692},"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"JY-20915","depth":14,"bounds":{"left":0.53357714,"top":0.0,"width":0.019115692,"height":0.011572227},"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915","depth":15,"bounds":{"left":0.53357714,"top":0.0,"width":0.019115692,"height":0.011572227},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"add sms received test","depth":14,"bounds":{"left":0.55502,"top":0.0,"width":0.050531916,"height":0.011572227},"on_screen":false,"help_text":"JY-20915 add sms received test","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false}]...
|
194585855194509247
|
-1048433147529382120
|
idle
|
accessibility
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo
Share & export
Copy
Show more options
Ask Gemini
Ask Gemini
Upload & tools
Microphone
Send message
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Summarize page
Summarize page
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
All issues(g then i)
All pull requests
All repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Pull requests (32)
Pull requests
(
32
)
Agents
Agents
Actions
Actions
Wiki
Wiki
Security and quality
Security and quality
Insights
Insights
Settings
Settings
JY-20915 add alias for EU #12105 Edit title
JY-20915 add alias for EU
#
12105
Edit title
Awaiting approval
Awaiting approval
Code
Code
Open
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Lines changed: 365 additions & 1 deletion
Conversation (1)
Conversation
(
1
)
Commits (5)
Commits
(
5
)
Checks (3)
Checks
(
3
)
Files changed (4)
Files changed
(
4
)
Open
JY-20915 add alias for EU #12105 LakyLak wants to merge 5 commits into master from JY-20915-add-domain-specific-email-text-relay Copy head branch name to clipboard
JY-20915 add alias for EU
JY-20915 add alias for EU
#
12105
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Conversation
Conversation
@LakyLak
Show options
LakyLak commented yesterday
LakyLak
LakyLak
commented
yesterday
yesterday
JIRA: JY-20915
JIRA:
JY-20915
JY-20915
Changes:
Changes:
Support text relay for EU
Prevent errors from EU on US
Add or remove reactions
LakyLak
LakyLak
added
5
commits
yesterday
yesterday
@LakyLak
JY-20915
JY-20915
add alias for EU
add alias for EU
8 / 10 checks OK
e5255d5
e5255d5
@LakyLak
JY-20915
JY-20915
fix env recognization
fix env recognization
11 / 12 checks OK
8953054
8953054
@LakyLak
JY-20915
JY-20915
code smells & tests
code smells & tests
11 / 12 checks OK
73d4bea
73d4bea
@LakyLak
JY-20915
JY-20915
add more tests
add more tests
11 / 12 checks OK
5d2dd3e
5d2dd3e
@LakyLak
JY-20915
JY-20915
add sms received test...
|
NULL
|
NULL
|
NULL
|
NULL
|
|
65251
|
NULL
|
0
|
2026-05-20T15:34:50.131666+00:00
|
/Users/lukas/.screenpipe/data/data/2026-05-20/1779 /Users/lukas/.screenpipe/data/data/2026-05-20/1779291290131_m1.jpg...
|
Firefox
|
JY-20915 add alias for EU by LakyLak · Pull Reques JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app — Work...
|
1
|
github.com/jiminny/app/pull/12105
|
monitor_1
|
NULL
|
NULL
|
NULL
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo
Share & export
Copy
Show more options
Ask Gemini
Ask Gemini
Upload & tools
Microphone
Send message
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Summarize page
Summarize page
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
All issues(g then i)
All pull requests
All repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Pull requests (32)
Pull requests
(
32
)
Agents
Agents
Actions
Actions
Wiki
Wiki
Security and quality
Security and quality
Insights
Insights
Settings
Settings
JY-20915 add alias for EU #12105 Edit title
JY-20915 add alias for EU
#
12105
Edit title
Awaiting approval
Awaiting approval
Code
Code
Open
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Lines changed: 365 additions & 1 deletion
Conversation (1)
Conversation
(
1
)
Commits (5)
Commits
(
5
)
Checks (3)
Checks
(
3
)
Files changed (4)
Files changed
(
4
)
Open
JY-20915 add alias for EU #12105 LakyLak wants to merge 5 commits into master from JY-20915-add-domain-specific-email-text-relay Copy head branch name to clipboard
JY-20915 add alias for EU
JY-20915 add alias for EU
#
12105
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Conversation
Conversation
@LakyLak
Show options
LakyLak commented yesterday
LakyLak
LakyLak
commented
yesterday
yesterday
JIRA: JY-20915
JIRA:
JY-20915
JY-20915
Changes:
Changes:
Support text relay for EU
Prevent errors from EU on US
Add or remove reactions
LakyLak
LakyLak
added
5
commits
yesterday
yesterday
@LakyLak
JY-20915
JY-20915
add alias for EU
add alias for EU
8 / 10 checks OK
e5255d5
e5255d5
@LakyLak
JY-20915
JY-20915
fix env recognization
fix env recognization
11 / 12 checks OK
8953054
8953054
@LakyLak
JY-20915
JY-20915
code smells & tests
code smells & tests
11 / 12 checks OK
73d4bea
73d4bea
@LakyLak
JY-20915
JY-20915
add more tests
add more tests
11 / 12 checks OK
5d2dd3e...
|
[{"role":"AXRadioButton","text [{"role":"AXRadioButton","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Close tab","depth":5,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXRadioButton","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Pipelines - jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Pipelines - jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Tab","depth":4,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Customize sidebar","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Close Google Gemini (⌃X)","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Tabs from other devices","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open history (⇧⌘H)","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open bookmarks (⌘B)","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"AI Chat settings","depth":7,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Close","depth":7,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Main menu","depth":13,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"New chat","depth":12,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Gemini","depth":15,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Chat","depth":13,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Open menu for conversation actions.","depth":13,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Conversation with Gemini","depth":15,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Conversation with Gemini","depth":16,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"You said I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.","depth":23,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Expand","depth":21,"on_screen":false,"role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a concise summary of the changes made in the code selection:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Refactoring and Dependencies","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Refactoring and Dependencies","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Replaced Interface with Implementation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Changed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusInterface","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusManager","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"in the imports and constructor dependency injection.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Redis Support:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Imported","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Illuminate\\Support\\Facades\\Redis","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to handle a newly introduced memory-logging lock mechanism.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Traits:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"WorkerIdTrait","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", along with the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"worker","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"field tracking in the final logs.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Loop and Worker Lifecycle Adjustments","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Loop and Worker Lifecycle Adjustments","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Worker Auto-Restart:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Deleted the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property, the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"constant, and the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"shouldIterateAgain()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method. The script no longer tracks execution iterations or triggers a self-managed worker restart.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Simplified Chunk Method Return:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Modified","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to always return","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Priority Handling for Throttled Entities","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Priority Handling for Throttled Entities","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Stale Record Elevation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Updated the logic in","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"rescheduleThrottledEntities()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to check if a throttled entity group is classified as \"stale\". If it is, the entities are rescheduled using","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"QueuePriorityEnum::NORMAL","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"; otherwise, they keep their respective queue priority.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Metrics, Logging, and Environment Handling","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Metrics, Logging, and Environment Handling","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Environment Tracking:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Introduced a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$environment","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property (defaulting to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"'local'","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") and a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"setEnvironment()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method to dynamically assign it.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Memory Monitoring via Datadog Gauge:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method that submits peak memory allocation to Datadog as a gauge metric (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"jiminny.reindex-memory-usage","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis Lock for Metrics Throttling:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Implemented a Redis-backed lock (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"%s-process-manager-lock","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") with a 60-second expiration inside","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to prevent over-reporting metrics.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Log Payload Update:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added total processed items (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$updatedDocuments + $deletedDocuments","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") to the final information log.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":23,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"You said are there any issues or potential bottlenecks","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"are there any issues or potential bottlenecks","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Edit","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a breakdown of the potential issues to look out for:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"1. Memory Leak Risk (OOM Crashes)","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"1. Memory Leak Risk (OOM Crashes)","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Restart Logic:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The most significant change is the removal of the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Dependency Leaks:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If any underlying service (like the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ElasticSearchClient","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", ORM, or Datadog helper) retains references to processed entities,","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Hardcoded Limits:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The code still includes","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ini_set('memory_limit', '512M');","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":". Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"2. Race Condition in Redis Lock","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"2. Race Condition in Redis Lock","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Non-Atomic Operations:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method uses three distinct steps to handle the lock:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"get()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":",","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", and","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"expire()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If multiple workers process the same","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$this->updateTarget","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"concurrently, they can both pass the","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"if (! is_null(Redis::get(...)))","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"check at the exact same millisecond. Both will then execute","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and push duplicate metrics to Datadog.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Fix:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"This should be replaced with a single atomic operation using Redis's","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"NX","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Not eXists) and","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"EX","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Expire) flags, or Laravel’s built-in atomic cache locks:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis::set($gaugeLockName, true, 'EX', 60, 'NX')","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"3. Infinite Loops and Caller Contract","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"3. Infinite Loops and Caller Contract","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Return Value Change:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"previously returned whether it should iterate again based on staleness or chunk size. It now always returns","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(unless the selection is totally empty).","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the code calling this method relies on a","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"false","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"return to pause, sleep, or gracefully terminate, changing this to an unconditional","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"could result in an aggressive \"busy loop\" that spikes CPU usage.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"4. Performance Overhead","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"4. Performance Overhead","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Expensive Garbage Collection:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Calling","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles();","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"on","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"every single chunk","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Synchronous Metric Dispatch:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::increment","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"or","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::gauge","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"5. Lack of Explicit Error Handling","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5. Lack of Explicit Error Handling","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"No","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"try/catch","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Blocks:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The script executes complex network operations (","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"loadDocuments","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":22,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":22,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Redo","depth":22,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":21,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":22,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":21,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXTextArea","text":"Ask Gemini","depth":21,"on_screen":true,"value":"Ask Gemini","help_text":"","role_description":"text entry area","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Ask Gemini","depth":22,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Upload & tools","depth":20,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Microphone","depth":20,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Send message","depth":20,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":false,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.","depth":17,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Your privacy & Gemini Opens in a new window","depth":17,"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Your privacy & Gemini","depth":18,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Opens in a new window","depth":19,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Summarize page","depth":7,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Summarize page","depth":9,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Skip to content","depth":6,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Skip to content","depth":7,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Open menu","depth":10,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"Homepage (g then d)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"jiminny","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"jiminny","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"app","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"app","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Search or jump to…","depth":9,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Type","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"/","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to search","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Chat with Copilot","depth":10,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXMenuButton","text":"Open Copilot…","depth":9,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXMenuButton","text":"Create new...","depth":9,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"All issues(g then i)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"All pull requests","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"All repositories","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"You have unread notifications(g then n)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Open user navigation menu","depth":9,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Repository navigation","depth":9,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Repository navigation","depth":10,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Code","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Code","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Pull requests (32)","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Pull requests","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"32","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Agents","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Agents","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Actions","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Actions","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Wiki","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Wiki","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Security and quality","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Security and quality","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Insights","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Insights","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Settings","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Settings","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"JY-20915 add alias for EU #12105 Edit title","depth":13,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"JY-20915 add alias for EU","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"#","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"12105","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Edit title","depth":14,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Awaiting approval","depth":13,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Awaiting approval","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXMenuButton","text":"Code","depth":13,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXStaticText","text":"Code","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Open","depth":13,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"LakyLak","depth":15,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"wants to merge 5 commits into","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"master","depth":15,"on_screen":false,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"master","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"from","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915-add-domain-specific-email-text-relay","depth":16,"on_screen":false,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915-add-domain-specific-email-text-relay","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy head branch name to clipboard","depth":16,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Lines changed: 365 additions & 1 deletion","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Conversation (1)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"Conversation","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"1","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Commits (5)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Commits","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Checks (3)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Checks","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"3","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Files changed (4)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Files changed","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"4","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Open","depth":14,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"JY-20915 add alias for EU #12105 LakyLak wants to merge 5 commits into master from JY-20915-add-domain-specific-email-text-relay Copy head branch name to clipboard","depth":14,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915 add alias for EU","depth":16,"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915 add alias for EU","depth":17,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"#","depth":16,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"12105","depth":16,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"LakyLak","depth":18,"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":19,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"wants to merge 5 commits into","depth":18,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"master","depth":18,"on_screen":true,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"master","depth":19,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"from","depth":19,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915-add-domain-specific-email-text-relay","depth":19,"on_screen":true,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915-add-domain-specific-email-text-relay","depth":20,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy head branch name to clipboard","depth":19,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Conversation","depth":12,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Conversation","depth":13,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@LakyLak","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show options","depth":15,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXHeading","text":"LakyLak commented yesterday","depth":14,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXLink","text":"LakyLak","depth":16,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"commented","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"yesterday","depth":15,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"yesterday","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"JIRA: JY-20915","depth":16,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"JIRA:","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915","depth":17,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Changes:","depth":16,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Changes:","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Support text relay for EU","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Prevent errors from EU on US","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Add or remove reactions","depth":16,"on_screen":false,"help_text":"","role_description":"summary","subrole":"AXSummary","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"LakyLak","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"added","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"commits","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"yesterday","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"yesterday","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@LakyLak","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"JY-20915","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"add alias for EU","depth":14,"on_screen":false,"help_text":"JY-20915 add alias for EU","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"add alias for EU","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"8 / 10 checks OK","depth":14,"on_screen":false,"help_text":"","role_description":"summary","subrole":"AXSummary","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"e5255d5","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"e5255d5","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@LakyLak","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"JY-20915","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"fix env recognization","depth":14,"on_screen":false,"help_text":"JY-20915 fix env recognization","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"fix env recognization","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"11 / 12 checks OK","depth":14,"on_screen":false,"help_text":"","role_description":"summary","subrole":"AXSummary","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"8953054","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"8953054","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@LakyLak","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"JY-20915","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"code smells & tests","depth":14,"on_screen":false,"help_text":"JY-20915 code smells & tests","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"code smells & tests","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"11 / 12 checks OK","depth":14,"on_screen":false,"help_text":"","role_description":"summary","subrole":"AXSummary","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"73d4bea","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"73d4bea","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@LakyLak","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"JY-20915","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"add more tests","depth":14,"on_screen":false,"help_text":"JY-20915 add more tests","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"add more tests","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"11 / 12 checks OK","depth":14,"on_screen":false,"help_text":"","role_description":"summary","subrole":"AXSummary","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"5d2dd3e","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false}]...
|
-7101349700791160676
|
-1048293509552638184
|
idle
|
accessibility
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo
Share & export
Copy
Show more options
Ask Gemini
Ask Gemini
Upload & tools
Microphone
Send message
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Summarize page
Summarize page
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
All issues(g then i)
All pull requests
All repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Pull requests (32)
Pull requests
(
32
)
Agents
Agents
Actions
Actions
Wiki
Wiki
Security and quality
Security and quality
Insights
Insights
Settings
Settings
JY-20915 add alias for EU #12105 Edit title
JY-20915 add alias for EU
#
12105
Edit title
Awaiting approval
Awaiting approval
Code
Code
Open
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Lines changed: 365 additions & 1 deletion
Conversation (1)
Conversation
(
1
)
Commits (5)
Commits
(
5
)
Checks (3)
Checks
(
3
)
Files changed (4)
Files changed
(
4
)
Open
JY-20915 add alias for EU #12105 LakyLak wants to merge 5 commits into master from JY-20915-add-domain-specific-email-text-relay Copy head branch name to clipboard
JY-20915 add alias for EU
JY-20915 add alias for EU
#
12105
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Conversation
Conversation
@LakyLak
Show options
LakyLak commented yesterday
LakyLak
LakyLak
commented
yesterday
yesterday
JIRA: JY-20915
JIRA:
JY-20915
JY-20915
Changes:
Changes:
Support text relay for EU
Prevent errors from EU on US
Add or remove reactions
LakyLak
LakyLak
added
5
commits
yesterday
yesterday
@LakyLak
JY-20915
JY-20915
add alias for EU
add alias for EU
8 / 10 checks OK
e5255d5
e5255d5
@LakyLak
JY-20915
JY-20915
fix env recognization
fix env recognization
11 / 12 checks OK
8953054
8953054
@LakyLak
JY-20915
JY-20915
code smells & tests
code smells & tests
11 / 12 checks OK
73d4bea
73d4bea
@LakyLak
JY-20915
JY-20915
add more tests
add more tests
11 / 12 checks OK
5d2dd3e...
|
NULL
|
NULL
|
NULL
|
NULL
|
|
65252
|
2290
|
0
|
2026-05-20T15:35:02.028892+00:00
|
/Users/lukas/.screenpipe/data/data/2026-05-20/1779 /Users/lukas/.screenpipe/data/data/2026-05-20/1779291302028_m2.jpg...
|
Firefox
|
JY-20915 add alias for EU by LakyLak · Pull Reques JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app — Work...
|
1
|
github.com/jiminny/app/pull/12105
|
monitor_2
|
NULL
|
NULL
|
NULL
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo
Share & export
Copy
Show more options
Ask Gemini
Ask Gemini
Upload & tools...
|
[{"role":"AXRadioButton","text [{"role":"AXRadioButton","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":4,"bounds":{"left":0.0,"top":0.0518755,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.06304868,"width":0.10106383,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":4,"bounds":{"left":0.0,"top":0.08459697,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.09577015,"width":0.10721409,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.11731844,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.12849163,"width":0.15791224,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.15003991,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.16121309,"width":0.1796875,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.18276137,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.19393456,"width":0.12699468,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Close tab","depth":5,"bounds":{"left":0.06732048,"top":0.18994413,"width":0.007978723,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXRadioButton","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":4,"bounds":{"left":0.0,"top":0.21548285,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":5,"bounds":{"left":0.013297873,"top":0.22665602,"width":0.10139628,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":4,"bounds":{"left":0.0,"top":0.2482043,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.25937748,"width":0.18068483,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":4,"bounds":{"left":0.0,"top":0.28092578,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":5,"bounds":{"left":0.013297873,"top":0.29209897,"width":0.4644282,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Pipelines - jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.31364724,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Pipelines - jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.32482043,"width":0.039228722,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.3463687,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.3575419,"width":0.15824468,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Tab","depth":4,"bounds":{"left":0.0028257978,"top":0.38068634,"width":0.07413564,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Customize sidebar","depth":6,"bounds":{"left":0.0028257978,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Close Google Gemini (⌃X)","depth":6,"bounds":{"left":0.013796543,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Tabs from other devices","depth":6,"bounds":{"left":0.024933511,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open history (⇧⌘H)","depth":6,"bounds":{"left":0.036070477,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open bookmarks (⌘B)","depth":6,"bounds":{"left":0.04720745,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"AI Chat settings","depth":7,"bounds":{"left":0.35854387,"top":0.055067837,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Close","depth":7,"bounds":{"left":0.37051198,"top":0.055067837,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Main menu","depth":13,"bounds":{"left":0.08228058,"top":0.09736632,"width":0.015957447,"height":0.03830806},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"New chat","depth":12,"bounds":{"left":0.09823803,"top":0.10215483,"width":0.028091755,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Gemini","depth":15,"bounds":{"left":0.1008976,"top":0.10654429,"width":0.02144282,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Chat","depth":13,"bounds":{"left":0.35854387,"top":0.10215483,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Open menu for conversation actions.","depth":13,"bounds":{"left":0.37051198,"top":0.10215483,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Conversation with Gemini","depth":15,"bounds":{"left":0.079288565,"top":0.14445332,"width":0.0003324468,"height":0.0007980846},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Conversation with Gemini","depth":16,"bounds":{"left":0.079288565,"top":0.14684756,"width":0.1200133,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"You said I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.","depth":23,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Expand","depth":21,"on_screen":false,"role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a concise summary of the changes made in the code selection:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Refactoring and Dependencies","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Refactoring and Dependencies","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Replaced Interface with Implementation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Changed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusInterface","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusManager","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"in the imports and constructor dependency injection.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Redis Support:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Imported","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Illuminate\\Support\\Facades\\Redis","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to handle a newly introduced memory-logging lock mechanism.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Traits:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"WorkerIdTrait","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", along with the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"worker","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"field tracking in the final logs.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Loop and Worker Lifecycle Adjustments","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Loop and Worker Lifecycle Adjustments","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Worker Auto-Restart:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Deleted the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property, the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"constant, and the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"shouldIterateAgain()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method. The script no longer tracks execution iterations or triggers a self-managed worker restart.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Simplified Chunk Method Return:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Modified","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to always return","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Priority Handling for Throttled Entities","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Priority Handling for Throttled Entities","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Stale Record Elevation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Updated the logic in","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"rescheduleThrottledEntities()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to check if a throttled entity group is classified as \"stale\". If it is, the entities are rescheduled using","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"QueuePriorityEnum::NORMAL","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"; otherwise, they keep their respective queue priority.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Metrics, Logging, and Environment Handling","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Metrics, Logging, and Environment Handling","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Environment Tracking:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Introduced a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$environment","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property (defaulting to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"'local'","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") and a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"setEnvironment()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method to dynamically assign it.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Memory Monitoring via Datadog Gauge:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method that submits peak memory allocation to Datadog as a gauge metric (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"jiminny.reindex-memory-usage","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis Lock for Metrics Throttling:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Implemented a Redis-backed lock (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"%s-process-manager-lock","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") with a 60-second expiration inside","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to prevent over-reporting metrics.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Log Payload Update:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added total processed items (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$updatedDocuments + $deletedDocuments","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") to the final information log.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":23,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"You said are there any issues or potential bottlenecks","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"are there any issues or potential bottlenecks","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Edit","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a breakdown of the potential issues to look out for:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"1. Memory Leak Risk (OOM Crashes)","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"1. Memory Leak Risk (OOM Crashes)","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Restart Logic:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The most significant change is the removal of the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Dependency Leaks:","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.050033245,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If any underlying service (like the","depth":26,"bounds":{"left":0.17453457,"top":0.0,"width":0.08277926,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ElasticSearchClient","depth":27,"bounds":{"left":0.25930852,"top":0.0,"width":0.056848403,"height":0.015163607},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", ORM, or Datadog helper) retains references to processed entities,","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.21775267,"height":0.03631285},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"bounds":{"left":0.26811835,"top":0.0,"width":0.056848403,"height":0.015163607},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.23138298,"height":0.05546688},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Hardcoded Limits:","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.047041222,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The code still includes","depth":26,"bounds":{"left":0.17154256,"top":0.0,"width":0.05651596,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ini_set('memory_limit', '512M');","depth":27,"bounds":{"left":0.23005319,"top":0.0,"width":0.095744684,"height":0.015163607},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":". Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.23055187,"height":0.05546688},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"2. Race Condition in Redis Lock","depth":23,"bounds":{"left":0.111369684,"top":0.07940942,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"2. Race Condition in Redis Lock","depth":24,"bounds":{"left":0.111369684,"top":0.079010375,"width":0.09125665,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Non-Atomic Operations:","depth":26,"bounds":{"left":0.12450133,"top":0.10614525,"width":0.062333778,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The","depth":26,"bounds":{"left":0.18683511,"top":0.10614525,"width":0.011635638,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage","depth":27,"bounds":{"left":0.20046543,"top":0.10774142,"width":0.041888297,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method uses three distinct steps to handle the lock:","depth":26,"bounds":{"left":0.12450133,"top":0.10614525,"width":0.22722739,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"get()","depth":27,"bounds":{"left":0.14860372,"top":0.12689546,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":",","depth":26,"bounds":{"left":0.16555852,"top":0.12529927,"width":0.002493351,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"bounds":{"left":0.17004654,"top":0.12689546,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", and","depth":26,"bounds":{"left":0.18700133,"top":0.12529927,"width":0.013131649,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"expire()","depth":27,"bounds":{"left":0.20212767,"top":0.12689546,"width":0.023936171,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"bounds":{"left":0.22805852,"top":0.12529927,"width":0.0013297872,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"bounds":{"left":0.12450133,"top":0.15403032,"width":0.040059842,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If multiple workers process the same","depth":26,"bounds":{"left":0.16456117,"top":0.15403032,"width":0.09275266,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$this->updateTarget","depth":27,"bounds":{"left":0.25930852,"top":0.15562649,"width":0.056848403,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"concurrently, they can both pass the","depth":26,"bounds":{"left":0.12450133,"top":0.15403032,"width":0.22855718,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"if (! is_null(Redis::get(...)))","depth":27,"bounds":{"left":0.18351063,"top":0.17478053,"width":0.09275266,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"check at the exact same millisecond. Both will then execute","depth":26,"bounds":{"left":0.12450133,"top":0.17318435,"width":0.21575798,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"bounds":{"left":0.2122673,"top":0.19393456,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and push duplicate metrics to Datadog.","depth":26,"bounds":{"left":0.22922207,"top":0.19233839,"width":0.09823803,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Fix:","depth":26,"bounds":{"left":0.12450133,"top":0.22106944,"width":0.019281914,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"This should be replaced with a single atomic operation using Redis's","depth":26,"bounds":{"left":0.14378324,"top":0.22106944,"width":0.16821809,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"NX","depth":27,"bounds":{"left":0.31399602,"top":0.22266561,"width":0.005984043,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Not eXists) and","depth":26,"bounds":{"left":0.12450133,"top":0.22106944,"width":0.22855718,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"EX","depth":27,"bounds":{"left":0.13696809,"top":0.24181964,"width":0.005984043,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Expire) flags, or Laravel’s built-in atomic cache locks:","depth":26,"bounds":{"left":0.14494681,"top":0.24022347,"width":0.13264628,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis::set($gaugeLockName, true, 'EX', 60, 'NX')","depth":27,"bounds":{"left":0.12649602,"top":0.26097366,"width":0.14361702,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"bounds":{"left":0.27210772,"top":0.25937748,"width":0.0013297872,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"3. Infinite Loops and Caller Contract","depth":23,"bounds":{"left":0.111369684,"top":0.30925778,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"3. Infinite Loops and Caller Contract","depth":24,"bounds":{"left":0.111369684,"top":0.30885875,"width":0.10571808,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Return Value Change:","depth":26,"bounds":{"left":0.12450133,"top":0.33599362,"width":0.055352394,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"bounds":{"left":0.18301196,"top":0.33758977,"width":0.06881649,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"previously returned whether it should iterate again based on staleness or chunk size. It now always returns","depth":26,"bounds":{"left":0.12450133,"top":0.33599362,"width":0.2237367,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"bounds":{"left":0.29438165,"top":0.3567438,"width":0.011968086,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(unless the selection is totally empty).","depth":26,"bounds":{"left":0.12450133,"top":0.35514766,"width":0.2130984,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"bounds":{"left":0.12450133,"top":0.40303272,"width":0.040059842,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the code calling this method relies on a","depth":26,"bounds":{"left":0.16456117,"top":0.40303272,"width":0.10305851,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"false","depth":27,"bounds":{"left":0.26961437,"top":0.4046289,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"return to pause, sleep, or gracefully terminate, changing this to an unconditional","depth":26,"bounds":{"left":0.12450133,"top":0.40303272,"width":0.22589761,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"bounds":{"left":0.26163563,"top":0.4237829,"width":0.011968086,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"could result in an aggressive \"busy loop\" that spikes CPU usage.","depth":26,"bounds":{"left":0.12450133,"top":0.42218676,"width":0.22290559,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"4. Performance Overhead","depth":23,"bounds":{"left":0.111369684,"top":0.49122107,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"4. Performance Overhead","depth":24,"bounds":{"left":0.111369684,"top":0.49082202,"width":0.076130316,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Expensive Garbage Collection:","depth":26,"bounds":{"left":0.12450133,"top":0.5179569,"width":0.078457445,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Calling","depth":26,"bounds":{"left":0.20295878,"top":0.5179569,"width":0.019281914,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles();","depth":27,"bounds":{"left":0.22423537,"top":0.51955307,"width":0.059840426,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"on","depth":26,"bounds":{"left":0.28607047,"top":0.5179569,"width":0.00880984,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"every single chunk","depth":26,"bounds":{"left":0.29488033,"top":0.5179569,"width":0.045545213,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.","depth":26,"bounds":{"left":0.12450133,"top":0.5179569,"width":0.23420878,"height":0.05546688},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Synchronous Metric Dispatch:","depth":26,"bounds":{"left":0.12450133,"top":0.584996,"width":0.07712766,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the","depth":26,"bounds":{"left":0.20162898,"top":0.584996,"width":0.015292553,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::increment","depth":27,"bounds":{"left":0.21891622,"top":0.5865922,"width":0.053856384,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"or","depth":26,"bounds":{"left":0.27476728,"top":0.584996,"width":0.0078125,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::gauge","depth":27,"bounds":{"left":0.28457448,"top":0.5865922,"width":0.041888297,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.","depth":26,"bounds":{"left":0.12450133,"top":0.584996,"width":0.23420878,"height":0.05546688},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"5. Lack of Explicit Error Handling","depth":23,"bounds":{"left":0.111369684,"top":0.67318434,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5. Lack of Explicit Error Handling","depth":24,"bounds":{"left":0.111369684,"top":0.67278534,"width":0.094913565,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"No","depth":26,"bounds":{"left":0.12450133,"top":0.6999202,"width":0.008477394,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"try/catch","depth":27,"bounds":{"left":0.1349734,"top":0.7015164,"width":0.026928192,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Blocks:","depth":26,"bounds":{"left":0.16389628,"top":0.6999202,"width":0.019614361,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The script executes complex network operations (","depth":26,"bounds":{"left":0.18351063,"top":0.6999202,"width":0.12367021,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"loadDocuments","depth":27,"bounds":{"left":0.30917552,"top":0.7015164,"width":0.038896278,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.","depth":26,"bounds":{"left":0.12450133,"top":0.6999202,"width":0.22888963,"height":0.09377494},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":22,"bounds":{"left":0.1087101,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":22,"bounds":{"left":0.124667555,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Redo","depth":22,"bounds":{"left":0.140625,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":21,"bounds":{"left":0.15658244,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":22,"bounds":{"left":0.17253989,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":21,"bounds":{"left":0.18849733,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXTextArea","text":"Ask Gemini","depth":21,"bounds":{"left":0.14594415,"top":0.86632085,"width":0.17021276,"height":0.01915403},"on_screen":true,"value":"Ask Gemini","help_text":"","role_description":"text entry area","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Ask Gemini","depth":22,"bounds":{"left":0.14594415,"top":0.8667199,"width":0.030418882,"height":0.018355945},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Upload & tools","depth":20,"bounds":{"left":0.12732713,"top":0.8567438,"width":0.015957447,"height":0.03830806},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false}]...
|
-2102630698172157269
|
-1070952279536486564
|
idle
|
accessibility
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo
Share & export
Copy
Show more options
Ask Gemini
Ask Gemini
Upload & tools...
|
65250
|
NULL
|
NULL
|
NULL
|
|
65253
|
2289
|
0
|
2026-05-20T15:35:20.490131+00:00
|
/Users/lukas/.screenpipe/data/data/2026-05-20/1779 /Users/lukas/.screenpipe/data/data/2026-05-20/1779291320490_m1.jpg...
|
Firefox
|
JY-20915 add alias for EU by LakyLak · Pull Reques JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app — Work...
|
1
|
github.com/jiminny/app/pull/12105
|
monitor_1
|
NULL
|
NULL
|
NULL
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo
Share & export
Copy
Show more options
Ask Gemini
Ask Gemini
Upload & tools
Microphone
Send message
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Summarize page
Summarize page
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type...
|
[{"role":"AXRadioButton","text [{"role":"AXRadioButton","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Close tab","depth":5,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXRadioButton","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Pipelines - jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Pipelines - jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Tab","depth":4,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Customize sidebar","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Close Google Gemini (⌃X)","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Tabs from other devices","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open history (⇧⌘H)","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open bookmarks (⌘B)","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"AI Chat settings","depth":7,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Close","depth":7,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Main menu","depth":13,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"New chat","depth":12,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Gemini","depth":15,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Chat","depth":13,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Open menu for conversation actions.","depth":13,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Conversation with Gemini","depth":15,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Conversation with Gemini","depth":16,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"You said I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.","depth":23,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Expand","depth":21,"on_screen":false,"role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a concise summary of the changes made in the code selection:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Refactoring and Dependencies","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Refactoring and Dependencies","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Replaced Interface with Implementation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Changed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusInterface","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusManager","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"in the imports and constructor dependency injection.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Redis Support:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Imported","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Illuminate\\Support\\Facades\\Redis","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to handle a newly introduced memory-logging lock mechanism.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Traits:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"WorkerIdTrait","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", along with the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"worker","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"field tracking in the final logs.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Loop and Worker Lifecycle Adjustments","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Loop and Worker Lifecycle Adjustments","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Worker Auto-Restart:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Deleted the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property, the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"constant, and the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"shouldIterateAgain()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method. The script no longer tracks execution iterations or triggers a self-managed worker restart.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Simplified Chunk Method Return:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Modified","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to always return","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Priority Handling for Throttled Entities","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Priority Handling for Throttled Entities","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Stale Record Elevation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Updated the logic in","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"rescheduleThrottledEntities()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to check if a throttled entity group is classified as \"stale\". If it is, the entities are rescheduled using","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"QueuePriorityEnum::NORMAL","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"; otherwise, they keep their respective queue priority.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Metrics, Logging, and Environment Handling","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Metrics, Logging, and Environment Handling","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Environment Tracking:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Introduced a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$environment","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property (defaulting to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"'local'","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") and a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"setEnvironment()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method to dynamically assign it.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Memory Monitoring via Datadog Gauge:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method that submits peak memory allocation to Datadog as a gauge metric (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"jiminny.reindex-memory-usage","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis Lock for Metrics Throttling:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Implemented a Redis-backed lock (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"%s-process-manager-lock","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") with a 60-second expiration inside","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to prevent over-reporting metrics.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Log Payload Update:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added total processed items (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$updatedDocuments + $deletedDocuments","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") to the final information log.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":23,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"You said are there any issues or potential bottlenecks","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"are there any issues or potential bottlenecks","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Edit","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a breakdown of the potential issues to look out for:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"1. Memory Leak Risk (OOM Crashes)","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"1. Memory Leak Risk (OOM Crashes)","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Restart Logic:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The most significant change is the removal of the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Dependency Leaks:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If any underlying service (like the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ElasticSearchClient","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", ORM, or Datadog helper) retains references to processed entities,","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Hardcoded Limits:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The code still includes","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ini_set('memory_limit', '512M');","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":". Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"2. Race Condition in Redis Lock","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"2. Race Condition in Redis Lock","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Non-Atomic Operations:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method uses three distinct steps to handle the lock:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"get()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":",","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", and","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"expire()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If multiple workers process the same","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$this->updateTarget","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"concurrently, they can both pass the","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"if (! is_null(Redis::get(...)))","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"check at the exact same millisecond. Both will then execute","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and push duplicate metrics to Datadog.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Fix:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"This should be replaced with a single atomic operation using Redis's","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"NX","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Not eXists) and","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"EX","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Expire) flags, or Laravel’s built-in atomic cache locks:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis::set($gaugeLockName, true, 'EX', 60, 'NX')","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"3. Infinite Loops and Caller Contract","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"3. Infinite Loops and Caller Contract","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Return Value Change:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"previously returned whether it should iterate again based on staleness or chunk size. It now always returns","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(unless the selection is totally empty).","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the code calling this method relies on a","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"false","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"return to pause, sleep, or gracefully terminate, changing this to an unconditional","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"could result in an aggressive \"busy loop\" that spikes CPU usage.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"4. Performance Overhead","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"4. Performance Overhead","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Expensive Garbage Collection:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Calling","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles();","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"on","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"every single chunk","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Synchronous Metric Dispatch:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::increment","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"or","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::gauge","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"5. Lack of Explicit Error Handling","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5. Lack of Explicit Error Handling","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"No","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"try/catch","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Blocks:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The script executes complex network operations (","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"loadDocuments","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":22,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":22,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Redo","depth":22,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":21,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":22,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":21,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXTextArea","text":"Ask Gemini","depth":21,"on_screen":true,"value":"Ask Gemini","help_text":"","role_description":"text entry area","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Ask Gemini","depth":22,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Upload & tools","depth":20,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Microphone","depth":20,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Send message","depth":20,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":false,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.","depth":17,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Your privacy & Gemini Opens in a new window","depth":17,"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Your privacy & Gemini","depth":18,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Opens in a new window","depth":19,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Summarize page","depth":7,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Summarize page","depth":9,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Skip to content","depth":6,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Skip to content","depth":7,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Open menu","depth":10,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"Homepage (g then d)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"jiminny","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"jiminny","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"app","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"app","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Search or jump to…","depth":9,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Type","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"}]...
|
8132219294423563179
|
-1070952245180926052
|
idle
|
accessibility
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo
Share & export
Copy
Show more options
Ask Gemini
Ask Gemini
Upload & tools
Microphone
Send message
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Summarize page
Summarize page
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type...
|
65251
|
NULL
|
NULL
|
NULL
|
|
65254
|
2290
|
1
|
2026-05-20T15:35:32.454067+00:00
|
/Users/lukas/.screenpipe/data/data/2026-05-20/1779 /Users/lukas/.screenpipe/data/data/2026-05-20/1779291332454_m2.jpg...
|
Firefox
|
JY-20915 add alias for EU by LakyLak · Pull Reques JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app — Work...
|
1
|
github.com/jiminny/app/pull/12105
|
monitor_2
|
NULL
|
NULL
|
NULL
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo
Share & export
Copy
Show more options...
|
[{"role":"AXRadioButton","text [{"role":"AXRadioButton","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":4,"bounds":{"left":0.0,"top":0.0518755,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.06304868,"width":0.10106383,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":4,"bounds":{"left":0.0,"top":0.08459697,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.09577015,"width":0.10721409,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.11731844,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.12849163,"width":0.15791224,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.15003991,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.16121309,"width":0.1796875,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.18276137,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.19393456,"width":0.12699468,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Close tab","depth":5,"bounds":{"left":0.06732048,"top":0.18994413,"width":0.007978723,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXRadioButton","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":4,"bounds":{"left":0.0,"top":0.21548285,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":5,"bounds":{"left":0.013297873,"top":0.22665602,"width":0.10139628,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":4,"bounds":{"left":0.0,"top":0.2482043,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.25937748,"width":0.18068483,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":4,"bounds":{"left":0.0,"top":0.28092578,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":5,"bounds":{"left":0.013297873,"top":0.29209897,"width":0.4644282,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Pipelines - jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.31364724,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Pipelines - jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.32482043,"width":0.039228722,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.3463687,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.3575419,"width":0.15824468,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Tab","depth":4,"bounds":{"left":0.0028257978,"top":0.38068634,"width":0.07413564,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Customize sidebar","depth":6,"bounds":{"left":0.0028257978,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Close Google Gemini (⌃X)","depth":6,"bounds":{"left":0.013796543,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Tabs from other devices","depth":6,"bounds":{"left":0.024933511,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open history (⇧⌘H)","depth":6,"bounds":{"left":0.036070477,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open bookmarks (⌘B)","depth":6,"bounds":{"left":0.04720745,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"AI Chat settings","depth":7,"bounds":{"left":0.35854387,"top":0.055067837,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Close","depth":7,"bounds":{"left":0.37051198,"top":0.055067837,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Main menu","depth":13,"bounds":{"left":0.08228058,"top":0.09736632,"width":0.015957447,"height":0.03830806},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"New chat","depth":12,"bounds":{"left":0.09823803,"top":0.10215483,"width":0.028091755,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Gemini","depth":15,"bounds":{"left":0.1008976,"top":0.10654429,"width":0.02144282,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Chat","depth":13,"bounds":{"left":0.35854387,"top":0.10215483,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Open menu for conversation actions.","depth":13,"bounds":{"left":0.37051198,"top":0.10215483,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Conversation with Gemini","depth":15,"bounds":{"left":0.079288565,"top":0.14445332,"width":0.0003324468,"height":0.0007980846},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Conversation with Gemini","depth":16,"bounds":{"left":0.079288565,"top":0.14684756,"width":0.1200133,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"You said I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.","depth":23,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Expand","depth":21,"on_screen":false,"role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a concise summary of the changes made in the code selection:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Refactoring and Dependencies","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Refactoring and Dependencies","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Replaced Interface with Implementation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Changed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusInterface","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusManager","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"in the imports and constructor dependency injection.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Redis Support:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Imported","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Illuminate\\Support\\Facades\\Redis","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to handle a newly introduced memory-logging lock mechanism.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Traits:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"WorkerIdTrait","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", along with the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"worker","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"field tracking in the final logs.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Loop and Worker Lifecycle Adjustments","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Loop and Worker Lifecycle Adjustments","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Worker Auto-Restart:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Deleted the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property, the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"constant, and the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"shouldIterateAgain()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method. The script no longer tracks execution iterations or triggers a self-managed worker restart.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Simplified Chunk Method Return:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Modified","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to always return","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Priority Handling for Throttled Entities","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Priority Handling for Throttled Entities","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Stale Record Elevation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Updated the logic in","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"rescheduleThrottledEntities()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to check if a throttled entity group is classified as \"stale\". If it is, the entities are rescheduled using","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"QueuePriorityEnum::NORMAL","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"; otherwise, they keep their respective queue priority.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Metrics, Logging, and Environment Handling","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Metrics, Logging, and Environment Handling","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Environment Tracking:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Introduced a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$environment","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property (defaulting to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"'local'","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") and a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"setEnvironment()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method to dynamically assign it.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Memory Monitoring via Datadog Gauge:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method that submits peak memory allocation to Datadog as a gauge metric (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"jiminny.reindex-memory-usage","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis Lock for Metrics Throttling:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Implemented a Redis-backed lock (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"%s-process-manager-lock","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") with a 60-second expiration inside","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to prevent over-reporting metrics.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Log Payload Update:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added total processed items (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$updatedDocuments + $deletedDocuments","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") to the final information log.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":23,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"You said are there any issues or potential bottlenecks","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"are there any issues or potential bottlenecks","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Edit","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a breakdown of the potential issues to look out for:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"1. Memory Leak Risk (OOM Crashes)","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"1. Memory Leak Risk (OOM Crashes)","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Restart Logic:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The most significant change is the removal of the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Dependency Leaks:","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.050033245,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If any underlying service (like the","depth":26,"bounds":{"left":0.17453457,"top":0.0,"width":0.08277926,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ElasticSearchClient","depth":27,"bounds":{"left":0.25930852,"top":0.0,"width":0.056848403,"height":0.015163607},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", ORM, or Datadog helper) retains references to processed entities,","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.21775267,"height":0.03631285},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"bounds":{"left":0.26811835,"top":0.0,"width":0.056848403,"height":0.015163607},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.23138298,"height":0.05546688},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Hardcoded Limits:","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.047041222,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The code still includes","depth":26,"bounds":{"left":0.17154256,"top":0.0,"width":0.05651596,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ini_set('memory_limit', '512M');","depth":27,"bounds":{"left":0.23005319,"top":0.0,"width":0.095744684,"height":0.015163607},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":". Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.23055187,"height":0.05546688},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"2. Race Condition in Redis Lock","depth":23,"bounds":{"left":0.111369684,"top":0.07940942,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"2. Race Condition in Redis Lock","depth":24,"bounds":{"left":0.111369684,"top":0.079010375,"width":0.09125665,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Non-Atomic Operations:","depth":26,"bounds":{"left":0.12450133,"top":0.10614525,"width":0.062333778,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The","depth":26,"bounds":{"left":0.18683511,"top":0.10614525,"width":0.011635638,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage","depth":27,"bounds":{"left":0.20046543,"top":0.10774142,"width":0.041888297,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method uses three distinct steps to handle the lock:","depth":26,"bounds":{"left":0.12450133,"top":0.10614525,"width":0.22722739,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"get()","depth":27,"bounds":{"left":0.14860372,"top":0.12689546,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":",","depth":26,"bounds":{"left":0.16555852,"top":0.12529927,"width":0.002493351,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"bounds":{"left":0.17004654,"top":0.12689546,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", and","depth":26,"bounds":{"left":0.18700133,"top":0.12529927,"width":0.013131649,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"expire()","depth":27,"bounds":{"left":0.20212767,"top":0.12689546,"width":0.023936171,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"bounds":{"left":0.22805852,"top":0.12529927,"width":0.0013297872,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"bounds":{"left":0.12450133,"top":0.15403032,"width":0.040059842,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If multiple workers process the same","depth":26,"bounds":{"left":0.16456117,"top":0.15403032,"width":0.09275266,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$this->updateTarget","depth":27,"bounds":{"left":0.25930852,"top":0.15562649,"width":0.056848403,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"concurrently, they can both pass the","depth":26,"bounds":{"left":0.12450133,"top":0.15403032,"width":0.22855718,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"if (! is_null(Redis::get(...)))","depth":27,"bounds":{"left":0.18351063,"top":0.17478053,"width":0.09275266,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"check at the exact same millisecond. Both will then execute","depth":26,"bounds":{"left":0.12450133,"top":0.17318435,"width":0.21575798,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"bounds":{"left":0.2122673,"top":0.19393456,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and push duplicate metrics to Datadog.","depth":26,"bounds":{"left":0.22922207,"top":0.19233839,"width":0.09823803,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Fix:","depth":26,"bounds":{"left":0.12450133,"top":0.22106944,"width":0.019281914,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"This should be replaced with a single atomic operation using Redis's","depth":26,"bounds":{"left":0.14378324,"top":0.22106944,"width":0.16821809,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"NX","depth":27,"bounds":{"left":0.31399602,"top":0.22266561,"width":0.005984043,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Not eXists) and","depth":26,"bounds":{"left":0.12450133,"top":0.22106944,"width":0.22855718,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"EX","depth":27,"bounds":{"left":0.13696809,"top":0.24181964,"width":0.005984043,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Expire) flags, or Laravel’s built-in atomic cache locks:","depth":26,"bounds":{"left":0.14494681,"top":0.24022347,"width":0.13264628,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis::set($gaugeLockName, true, 'EX', 60, 'NX')","depth":27,"bounds":{"left":0.12649602,"top":0.26097366,"width":0.14361702,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"bounds":{"left":0.27210772,"top":0.25937748,"width":0.0013297872,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"3. Infinite Loops and Caller Contract","depth":23,"bounds":{"left":0.111369684,"top":0.30925778,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"3. Infinite Loops and Caller Contract","depth":24,"bounds":{"left":0.111369684,"top":0.30885875,"width":0.10571808,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Return Value Change:","depth":26,"bounds":{"left":0.12450133,"top":0.33599362,"width":0.055352394,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"bounds":{"left":0.18301196,"top":0.33758977,"width":0.06881649,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"previously returned whether it should iterate again based on staleness or chunk size. It now always returns","depth":26,"bounds":{"left":0.12450133,"top":0.33599362,"width":0.2237367,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"bounds":{"left":0.29438165,"top":0.3567438,"width":0.011968086,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(unless the selection is totally empty).","depth":26,"bounds":{"left":0.12450133,"top":0.35514766,"width":0.2130984,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"bounds":{"left":0.12450133,"top":0.40303272,"width":0.040059842,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the code calling this method relies on a","depth":26,"bounds":{"left":0.16456117,"top":0.40303272,"width":0.10305851,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"false","depth":27,"bounds":{"left":0.26961437,"top":0.4046289,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"return to pause, sleep, or gracefully terminate, changing this to an unconditional","depth":26,"bounds":{"left":0.12450133,"top":0.40303272,"width":0.22589761,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"bounds":{"left":0.26163563,"top":0.4237829,"width":0.011968086,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"could result in an aggressive \"busy loop\" that spikes CPU usage.","depth":26,"bounds":{"left":0.12450133,"top":0.42218676,"width":0.22290559,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"4. Performance Overhead","depth":23,"bounds":{"left":0.111369684,"top":0.49122107,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"4. Performance Overhead","depth":24,"bounds":{"left":0.111369684,"top":0.49082202,"width":0.076130316,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Expensive Garbage Collection:","depth":26,"bounds":{"left":0.12450133,"top":0.5179569,"width":0.078457445,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Calling","depth":26,"bounds":{"left":0.20295878,"top":0.5179569,"width":0.019281914,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles();","depth":27,"bounds":{"left":0.22423537,"top":0.51955307,"width":0.059840426,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"on","depth":26,"bounds":{"left":0.28607047,"top":0.5179569,"width":0.00880984,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"every single chunk","depth":26,"bounds":{"left":0.29488033,"top":0.5179569,"width":0.045545213,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.","depth":26,"bounds":{"left":0.12450133,"top":0.5179569,"width":0.23420878,"height":0.05546688},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Synchronous Metric Dispatch:","depth":26,"bounds":{"left":0.12450133,"top":0.584996,"width":0.07712766,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the","depth":26,"bounds":{"left":0.20162898,"top":0.584996,"width":0.015292553,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::increment","depth":27,"bounds":{"left":0.21891622,"top":0.5865922,"width":0.053856384,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"or","depth":26,"bounds":{"left":0.27476728,"top":0.584996,"width":0.0078125,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::gauge","depth":27,"bounds":{"left":0.28457448,"top":0.5865922,"width":0.041888297,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.","depth":26,"bounds":{"left":0.12450133,"top":0.584996,"width":0.23420878,"height":0.05546688},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"5. Lack of Explicit Error Handling","depth":23,"bounds":{"left":0.111369684,"top":0.67318434,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5. Lack of Explicit Error Handling","depth":24,"bounds":{"left":0.111369684,"top":0.67278534,"width":0.094913565,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"No","depth":26,"bounds":{"left":0.12450133,"top":0.6999202,"width":0.008477394,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"try/catch","depth":27,"bounds":{"left":0.1349734,"top":0.7015164,"width":0.026928192,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Blocks:","depth":26,"bounds":{"left":0.16389628,"top":0.6999202,"width":0.019614361,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The script executes complex network operations (","depth":26,"bounds":{"left":0.18351063,"top":0.6999202,"width":0.12367021,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"loadDocuments","depth":27,"bounds":{"left":0.30917552,"top":0.7015164,"width":0.038896278,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.","depth":26,"bounds":{"left":0.12450133,"top":0.6999202,"width":0.22888963,"height":0.09377494},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":22,"bounds":{"left":0.1087101,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":22,"bounds":{"left":0.124667555,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Redo","depth":22,"bounds":{"left":0.140625,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":21,"bounds":{"left":0.15658244,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":22,"bounds":{"left":0.17253989,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":21,"bounds":{"left":0.18849733,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false}]...
|
2749669850961545648
|
-1070952279540680868
|
idle
|
accessibility
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo
Share & export
Copy
Show more options...
|
NULL
|
NULL
|
NULL
|
NULL
|
|
65255
|
2289
|
1
|
2026-05-20T15:35:50.871522+00:00
|
/Users/lukas/.screenpipe/data/data/2026-05-20/1779 /Users/lukas/.screenpipe/data/data/2026-05-20/1779291350871_m1.jpg...
|
Firefox
|
JY-20915 add alias for EU by LakyLak · Pull Reques JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app — Work...
|
1
|
github.com/jiminny/app/pull/12105
|
monitor_1
|
NULL
|
NULL
|
NULL
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo
Share & export
Copy
Show more options
Ask Gemini
Ask Gemini
Upload & tools
Microphone
Send message
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Summarize page
Summarize page
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
All issues(g then i)
All pull requests
All repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Pull requests (32)
Pull requests
(
32
)
Agents
Agents
Actions
Actions
Wiki
Wiki
Security and quality
Security and quality
Insights
Insights
Settings
Settings
JY-20915 add alias for EU #12105 Edit title
JY-20915 add alias for EU
#
12105
Edit title
Awaiting approval
Awaiting approval
Code
Code
Open
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Lines changed: 365 additions & 1 deletion
Conversation (1)
Conversation
(
1
)
Commits (5)
Commits
(
5
)
Checks (3)
Checks
(
3
)
Files changed (4)
Files changed
(
4
)
Open
JY-20915 add alias for EU #12105 LakyLak wants to merge 5 commits into master from JY-20915-add-domain-specific-email-text-relay Copy head branch name to clipboard
JY-20915 add alias for EU
JY-20915 add alias for EU
#
12105
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Conversation
Conversation
@LakyLak
Show options
LakyLak commented yesterday
LakyLak
LakyLak
commented
yesterday
yesterday
JIRA: JY-20915
JIRA:
JY-20915
JY-20915
Changes:
Changes:
Support text relay for EU
Prevent errors from EU on US
Add or remove reactions
LakyLak
LakyLak
added
5...
|
[{"role":"AXRadioButton","text [{"role":"AXRadioButton","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Close tab","depth":5,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXRadioButton","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Pipelines - jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Pipelines - jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Tab","depth":4,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Customize sidebar","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Close Google Gemini (⌃X)","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Tabs from other devices","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open history (⇧⌘H)","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open bookmarks (⌘B)","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"AI Chat settings","depth":7,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Close","depth":7,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Main menu","depth":13,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"New chat","depth":12,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Gemini","depth":15,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Chat","depth":13,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Open menu for conversation actions.","depth":13,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Conversation with Gemini","depth":15,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Conversation with Gemini","depth":16,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"You said I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.","depth":23,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Expand","depth":21,"on_screen":false,"role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a concise summary of the changes made in the code selection:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Refactoring and Dependencies","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Refactoring and Dependencies","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Replaced Interface with Implementation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Changed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusInterface","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusManager","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"in the imports and constructor dependency injection.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Redis Support:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Imported","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Illuminate\\Support\\Facades\\Redis","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to handle a newly introduced memory-logging lock mechanism.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Traits:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"WorkerIdTrait","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", along with the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"worker","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"field tracking in the final logs.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Loop and Worker Lifecycle Adjustments","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Loop and Worker Lifecycle Adjustments","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Worker Auto-Restart:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Deleted the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property, the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"constant, and the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"shouldIterateAgain()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method. The script no longer tracks execution iterations or triggers a self-managed worker restart.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Simplified Chunk Method Return:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Modified","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to always return","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Priority Handling for Throttled Entities","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Priority Handling for Throttled Entities","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Stale Record Elevation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Updated the logic in","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"rescheduleThrottledEntities()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to check if a throttled entity group is classified as \"stale\". If it is, the entities are rescheduled using","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"QueuePriorityEnum::NORMAL","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"; otherwise, they keep their respective queue priority.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Metrics, Logging, and Environment Handling","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Metrics, Logging, and Environment Handling","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Environment Tracking:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Introduced a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$environment","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property (defaulting to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"'local'","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") and a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"setEnvironment()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method to dynamically assign it.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Memory Monitoring via Datadog Gauge:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method that submits peak memory allocation to Datadog as a gauge metric (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"jiminny.reindex-memory-usage","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis Lock for Metrics Throttling:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Implemented a Redis-backed lock (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"%s-process-manager-lock","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") with a 60-second expiration inside","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to prevent over-reporting metrics.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Log Payload Update:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added total processed items (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$updatedDocuments + $deletedDocuments","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") to the final information log.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":23,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"You said are there any issues or potential bottlenecks","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"are there any issues or potential bottlenecks","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Edit","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a breakdown of the potential issues to look out for:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"1. Memory Leak Risk (OOM Crashes)","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"1. Memory Leak Risk (OOM Crashes)","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Restart Logic:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The most significant change is the removal of the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Dependency Leaks:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If any underlying service (like the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ElasticSearchClient","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", ORM, or Datadog helper) retains references to processed entities,","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Hardcoded Limits:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The code still includes","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ini_set('memory_limit', '512M');","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":". Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"2. Race Condition in Redis Lock","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"2. Race Condition in Redis Lock","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Non-Atomic Operations:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method uses three distinct steps to handle the lock:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"get()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":",","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", and","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"expire()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If multiple workers process the same","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$this->updateTarget","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"concurrently, they can both pass the","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"if (! is_null(Redis::get(...)))","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"check at the exact same millisecond. Both will then execute","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and push duplicate metrics to Datadog.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Fix:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"This should be replaced with a single atomic operation using Redis's","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"NX","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Not eXists) and","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"EX","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Expire) flags, or Laravel’s built-in atomic cache locks:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis::set($gaugeLockName, true, 'EX', 60, 'NX')","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"3. Infinite Loops and Caller Contract","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"3. Infinite Loops and Caller Contract","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Return Value Change:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"previously returned whether it should iterate again based on staleness or chunk size. It now always returns","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(unless the selection is totally empty).","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the code calling this method relies on a","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"false","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"return to pause, sleep, or gracefully terminate, changing this to an unconditional","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"could result in an aggressive \"busy loop\" that spikes CPU usage.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"4. Performance Overhead","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"4. Performance Overhead","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Expensive Garbage Collection:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Calling","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles();","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"on","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"every single chunk","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Synchronous Metric Dispatch:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::increment","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"or","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::gauge","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"5. Lack of Explicit Error Handling","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5. Lack of Explicit Error Handling","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"No","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"try/catch","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Blocks:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The script executes complex network operations (","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"loadDocuments","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":22,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":22,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Redo","depth":22,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":21,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":22,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":21,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXTextArea","text":"Ask Gemini","depth":21,"on_screen":true,"value":"Ask Gemini","help_text":"","role_description":"text entry area","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Ask Gemini","depth":22,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Upload & tools","depth":20,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Microphone","depth":20,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Send message","depth":20,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":false,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.","depth":17,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Your privacy & Gemini Opens in a new window","depth":17,"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Your privacy & Gemini","depth":18,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Opens in a new window","depth":19,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Summarize page","depth":7,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Summarize page","depth":9,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Skip to content","depth":6,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Skip to content","depth":7,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Open menu","depth":10,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"Homepage (g then d)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"jiminny","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"jiminny","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"app","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"app","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Search or jump to…","depth":9,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Type","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"/","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to search","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Chat with Copilot","depth":10,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXMenuButton","text":"Open Copilot…","depth":9,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXMenuButton","text":"Create new...","depth":9,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"All issues(g then i)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"All pull requests","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"All repositories","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"You have unread notifications(g then n)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Open user navigation menu","depth":9,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Repository navigation","depth":9,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Repository navigation","depth":10,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Code","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Code","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Pull requests (32)","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Pull requests","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"32","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Agents","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Agents","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Actions","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Actions","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Wiki","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Wiki","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Security and quality","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Security and quality","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Insights","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Insights","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Settings","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Settings","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"JY-20915 add alias for EU #12105 Edit title","depth":13,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"JY-20915 add alias for EU","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"#","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"12105","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Edit title","depth":14,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Awaiting approval","depth":13,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Awaiting approval","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXMenuButton","text":"Code","depth":13,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXStaticText","text":"Code","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Open","depth":13,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"LakyLak","depth":15,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"wants to merge 5 commits into","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"master","depth":15,"on_screen":false,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"master","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"from","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915-add-domain-specific-email-text-relay","depth":16,"on_screen":false,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915-add-domain-specific-email-text-relay","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy head branch name to clipboard","depth":16,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Lines changed: 365 additions & 1 deletion","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Conversation (1)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"Conversation","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"1","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Commits (5)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Commits","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Checks (3)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Checks","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"3","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Files changed (4)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Files changed","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"4","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Open","depth":14,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"JY-20915 add alias for EU #12105 LakyLak wants to merge 5 commits into master from JY-20915-add-domain-specific-email-text-relay Copy head branch name to clipboard","depth":14,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915 add alias for EU","depth":16,"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915 add alias for EU","depth":17,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"#","depth":16,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"12105","depth":16,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"LakyLak","depth":18,"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":19,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"wants to merge 5 commits into","depth":18,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"master","depth":18,"on_screen":true,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"master","depth":19,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"from","depth":19,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915-add-domain-specific-email-text-relay","depth":19,"on_screen":true,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915-add-domain-specific-email-text-relay","depth":20,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy head branch name to clipboard","depth":19,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Conversation","depth":12,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Conversation","depth":13,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@LakyLak","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show options","depth":15,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXHeading","text":"LakyLak commented yesterday","depth":14,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXLink","text":"LakyLak","depth":16,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"commented","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"yesterday","depth":15,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"yesterday","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"JIRA: JY-20915","depth":16,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"JIRA:","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915","depth":17,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Changes:","depth":16,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Changes:","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Support text relay for EU","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Prevent errors from EU on US","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Add or remove reactions","depth":16,"on_screen":false,"help_text":"","role_description":"summary","subrole":"AXSummary","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"LakyLak","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"added","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"}]...
|
-8573810667132866148
|
-1048292410041026792
|
idle
|
accessibility
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo
Share & export
Copy
Show more options
Ask Gemini
Ask Gemini
Upload & tools
Microphone
Send message
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Summarize page
Summarize page
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
All issues(g then i)
All pull requests
All repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Pull requests (32)
Pull requests
(
32
)
Agents
Agents
Actions
Actions
Wiki
Wiki
Security and quality
Security and quality
Insights
Insights
Settings
Settings
JY-20915 add alias for EU #12105 Edit title
JY-20915 add alias for EU
#
12105
Edit title
Awaiting approval
Awaiting approval
Code
Code
Open
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Lines changed: 365 additions & 1 deletion
Conversation (1)
Conversation
(
1
)
Commits (5)
Commits
(
5
)
Checks (3)
Checks
(
3
)
Files changed (4)
Files changed
(
4
)
Open
JY-20915 add alias for EU #12105 LakyLak wants to merge 5 commits into master from JY-20915-add-domain-specific-email-text-relay Copy head branch name to clipboard
JY-20915 add alias for EU
JY-20915 add alias for EU
#
12105
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Conversation
Conversation
@LakyLak
Show options
LakyLak commented yesterday
LakyLak
LakyLak
commented
yesterday
yesterday
JIRA: JY-20915
JIRA:
JY-20915
JY-20915
Changes:
Changes:
Support text relay for EU
Prevent errors from EU on US
Add or remove reactions
LakyLak
LakyLak
added
5...
|
NULL
|
NULL
|
NULL
|
NULL
|
|
65256
|
2290
|
2
|
2026-05-20T15:36:02.971077+00:00
|
/Users/lukas/.screenpipe/data/data/2026-05-20/1779 /Users/lukas/.screenpipe/data/data/2026-05-20/1779291362971_m2.jpg...
|
Firefox
|
JY-20915 add alias for EU by LakyLak · Pull Reques JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app — Work...
|
1
|
github.com/jiminny/app/pull/12105
|
monitor_2
|
NULL
|
NULL
|
NULL
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response...
|
[{"role":"AXRadioButton","text [{"role":"AXRadioButton","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":4,"bounds":{"left":0.0,"top":0.0518755,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.06304868,"width":0.10106383,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":4,"bounds":{"left":0.0,"top":0.08459697,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.09577015,"width":0.10721409,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.11731844,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.12849163,"width":0.15791224,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.15003991,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.16121309,"width":0.1796875,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.18276137,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.19393456,"width":0.12699468,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Close tab","depth":5,"bounds":{"left":0.06732048,"top":0.18994413,"width":0.007978723,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXRadioButton","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":4,"bounds":{"left":0.0,"top":0.21548285,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":5,"bounds":{"left":0.013297873,"top":0.22665602,"width":0.10139628,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":4,"bounds":{"left":0.0,"top":0.2482043,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.25937748,"width":0.18068483,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":4,"bounds":{"left":0.0,"top":0.28092578,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":5,"bounds":{"left":0.013297873,"top":0.29209897,"width":0.4644282,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Pipelines - jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.31364724,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Pipelines - jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.32482043,"width":0.039228722,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.3463687,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.3575419,"width":0.15824468,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Tab","depth":4,"bounds":{"left":0.0028257978,"top":0.38068634,"width":0.07413564,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Customize sidebar","depth":6,"bounds":{"left":0.0028257978,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Close Google Gemini (⌃X)","depth":6,"bounds":{"left":0.013796543,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Tabs from other devices","depth":6,"bounds":{"left":0.024933511,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open history (⇧⌘H)","depth":6,"bounds":{"left":0.036070477,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open bookmarks (⌘B)","depth":6,"bounds":{"left":0.04720745,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"AI Chat settings","depth":7,"bounds":{"left":0.35854387,"top":0.055067837,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Close","depth":7,"bounds":{"left":0.37051198,"top":0.055067837,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Main menu","depth":13,"bounds":{"left":0.08228058,"top":0.09736632,"width":0.015957447,"height":0.03830806},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"New chat","depth":12,"bounds":{"left":0.09823803,"top":0.10215483,"width":0.028091755,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Gemini","depth":15,"bounds":{"left":0.1008976,"top":0.10654429,"width":0.02144282,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Chat","depth":13,"bounds":{"left":0.35854387,"top":0.10215483,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Open menu for conversation actions.","depth":13,"bounds":{"left":0.37051198,"top":0.10215483,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Conversation with Gemini","depth":15,"bounds":{"left":0.079288565,"top":0.14445332,"width":0.0003324468,"height":0.0007980846},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Conversation with Gemini","depth":16,"bounds":{"left":0.079288565,"top":0.14684756,"width":0.1200133,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"You said I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.","depth":23,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Expand","depth":21,"on_screen":false,"role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a concise summary of the changes made in the code selection:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Refactoring and Dependencies","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Refactoring and Dependencies","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Replaced Interface with Implementation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Changed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusInterface","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusManager","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"in the imports and constructor dependency injection.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Redis Support:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Imported","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Illuminate\\Support\\Facades\\Redis","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to handle a newly introduced memory-logging lock mechanism.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Traits:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"WorkerIdTrait","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", along with the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"worker","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"field tracking in the final logs.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Loop and Worker Lifecycle Adjustments","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Loop and Worker Lifecycle Adjustments","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Worker Auto-Restart:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Deleted the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property, the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"constant, and the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"shouldIterateAgain()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method. The script no longer tracks execution iterations or triggers a self-managed worker restart.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Simplified Chunk Method Return:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Modified","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to always return","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Priority Handling for Throttled Entities","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Priority Handling for Throttled Entities","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Stale Record Elevation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Updated the logic in","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"rescheduleThrottledEntities()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to check if a throttled entity group is classified as \"stale\". If it is, the entities are rescheduled using","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"QueuePriorityEnum::NORMAL","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"; otherwise, they keep their respective queue priority.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Metrics, Logging, and Environment Handling","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Metrics, Logging, and Environment Handling","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Environment Tracking:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Introduced a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$environment","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property (defaulting to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"'local'","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") and a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"setEnvironment()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method to dynamically assign it.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Memory Monitoring via Datadog Gauge:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method that submits peak memory allocation to Datadog as a gauge metric (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"jiminny.reindex-memory-usage","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis Lock for Metrics Throttling:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Implemented a Redis-backed lock (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"%s-process-manager-lock","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") with a 60-second expiration inside","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to prevent over-reporting metrics.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Log Payload Update:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added total processed items (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$updatedDocuments + $deletedDocuments","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") to the final information log.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":23,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"You said are there any issues or potential bottlenecks","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"are there any issues or potential bottlenecks","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Edit","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a breakdown of the potential issues to look out for:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"1. Memory Leak Risk (OOM Crashes)","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"1. Memory Leak Risk (OOM Crashes)","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Restart Logic:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The most significant change is the removal of the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Dependency Leaks:","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.050033245,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If any underlying service (like the","depth":26,"bounds":{"left":0.17453457,"top":0.0,"width":0.08277926,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ElasticSearchClient","depth":27,"bounds":{"left":0.25930852,"top":0.0,"width":0.056848403,"height":0.015163607},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", ORM, or Datadog helper) retains references to processed entities,","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.21775267,"height":0.03631285},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"bounds":{"left":0.26811835,"top":0.0,"width":0.056848403,"height":0.015163607},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.23138298,"height":0.05546688},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Hardcoded Limits:","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.047041222,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The code still includes","depth":26,"bounds":{"left":0.17154256,"top":0.0,"width":0.05651596,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ini_set('memory_limit', '512M');","depth":27,"bounds":{"left":0.23005319,"top":0.0,"width":0.095744684,"height":0.015163607},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":". Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.23055187,"height":0.05546688},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"2. Race Condition in Redis Lock","depth":23,"bounds":{"left":0.111369684,"top":0.07940942,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"2. Race Condition in Redis Lock","depth":24,"bounds":{"left":0.111369684,"top":0.079010375,"width":0.09125665,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Non-Atomic Operations:","depth":26,"bounds":{"left":0.12450133,"top":0.10614525,"width":0.062333778,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The","depth":26,"bounds":{"left":0.18683511,"top":0.10614525,"width":0.011635638,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage","depth":27,"bounds":{"left":0.20046543,"top":0.10774142,"width":0.041888297,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method uses three distinct steps to handle the lock:","depth":26,"bounds":{"left":0.12450133,"top":0.10614525,"width":0.22722739,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"get()","depth":27,"bounds":{"left":0.14860372,"top":0.12689546,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":",","depth":26,"bounds":{"left":0.16555852,"top":0.12529927,"width":0.002493351,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"bounds":{"left":0.17004654,"top":0.12689546,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", and","depth":26,"bounds":{"left":0.18700133,"top":0.12529927,"width":0.013131649,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"expire()","depth":27,"bounds":{"left":0.20212767,"top":0.12689546,"width":0.023936171,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"bounds":{"left":0.22805852,"top":0.12529927,"width":0.0013297872,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"bounds":{"left":0.12450133,"top":0.15403032,"width":0.040059842,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If multiple workers process the same","depth":26,"bounds":{"left":0.16456117,"top":0.15403032,"width":0.09275266,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$this->updateTarget","depth":27,"bounds":{"left":0.25930852,"top":0.15562649,"width":0.056848403,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"concurrently, they can both pass the","depth":26,"bounds":{"left":0.12450133,"top":0.15403032,"width":0.22855718,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"if (! is_null(Redis::get(...)))","depth":27,"bounds":{"left":0.18351063,"top":0.17478053,"width":0.09275266,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"check at the exact same millisecond. Both will then execute","depth":26,"bounds":{"left":0.12450133,"top":0.17318435,"width":0.21575798,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"bounds":{"left":0.2122673,"top":0.19393456,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and push duplicate metrics to Datadog.","depth":26,"bounds":{"left":0.22922207,"top":0.19233839,"width":0.09823803,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Fix:","depth":26,"bounds":{"left":0.12450133,"top":0.22106944,"width":0.019281914,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"This should be replaced with a single atomic operation using Redis's","depth":26,"bounds":{"left":0.14378324,"top":0.22106944,"width":0.16821809,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"NX","depth":27,"bounds":{"left":0.31399602,"top":0.22266561,"width":0.005984043,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Not eXists) and","depth":26,"bounds":{"left":0.12450133,"top":0.22106944,"width":0.22855718,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"EX","depth":27,"bounds":{"left":0.13696809,"top":0.24181964,"width":0.005984043,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Expire) flags, or Laravel’s built-in atomic cache locks:","depth":26,"bounds":{"left":0.14494681,"top":0.24022347,"width":0.13264628,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis::set($gaugeLockName, true, 'EX', 60, 'NX')","depth":27,"bounds":{"left":0.12649602,"top":0.26097366,"width":0.14361702,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"bounds":{"left":0.27210772,"top":0.25937748,"width":0.0013297872,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"3. Infinite Loops and Caller Contract","depth":23,"bounds":{"left":0.111369684,"top":0.30925778,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"3. Infinite Loops and Caller Contract","depth":24,"bounds":{"left":0.111369684,"top":0.30885875,"width":0.10571808,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Return Value Change:","depth":26,"bounds":{"left":0.12450133,"top":0.33599362,"width":0.055352394,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"bounds":{"left":0.18301196,"top":0.33758977,"width":0.06881649,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"previously returned whether it should iterate again based on staleness or chunk size. It now always returns","depth":26,"bounds":{"left":0.12450133,"top":0.33599362,"width":0.2237367,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"bounds":{"left":0.29438165,"top":0.3567438,"width":0.011968086,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(unless the selection is totally empty).","depth":26,"bounds":{"left":0.12450133,"top":0.35514766,"width":0.2130984,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"bounds":{"left":0.12450133,"top":0.40303272,"width":0.040059842,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the code calling this method relies on a","depth":26,"bounds":{"left":0.16456117,"top":0.40303272,"width":0.10305851,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"false","depth":27,"bounds":{"left":0.26961437,"top":0.4046289,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"return to pause, sleep, or gracefully terminate, changing this to an unconditional","depth":26,"bounds":{"left":0.12450133,"top":0.40303272,"width":0.22589761,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"bounds":{"left":0.26163563,"top":0.4237829,"width":0.011968086,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"could result in an aggressive \"busy loop\" that spikes CPU usage.","depth":26,"bounds":{"left":0.12450133,"top":0.42218676,"width":0.22290559,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"4. Performance Overhead","depth":23,"bounds":{"left":0.111369684,"top":0.49122107,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"4. Performance Overhead","depth":24,"bounds":{"left":0.111369684,"top":0.49082202,"width":0.076130316,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Expensive Garbage Collection:","depth":26,"bounds":{"left":0.12450133,"top":0.5179569,"width":0.078457445,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Calling","depth":26,"bounds":{"left":0.20295878,"top":0.5179569,"width":0.019281914,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles();","depth":27,"bounds":{"left":0.22423537,"top":0.51955307,"width":0.059840426,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"on","depth":26,"bounds":{"left":0.28607047,"top":0.5179569,"width":0.00880984,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"every single chunk","depth":26,"bounds":{"left":0.29488033,"top":0.5179569,"width":0.045545213,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.","depth":26,"bounds":{"left":0.12450133,"top":0.5179569,"width":0.23420878,"height":0.05546688},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Synchronous Metric Dispatch:","depth":26,"bounds":{"left":0.12450133,"top":0.584996,"width":0.07712766,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the","depth":26,"bounds":{"left":0.20162898,"top":0.584996,"width":0.015292553,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::increment","depth":27,"bounds":{"left":0.21891622,"top":0.5865922,"width":0.053856384,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"or","depth":26,"bounds":{"left":0.27476728,"top":0.584996,"width":0.0078125,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::gauge","depth":27,"bounds":{"left":0.28457448,"top":0.5865922,"width":0.041888297,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.","depth":26,"bounds":{"left":0.12450133,"top":0.584996,"width":0.23420878,"height":0.05546688},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"5. Lack of Explicit Error Handling","depth":23,"bounds":{"left":0.111369684,"top":0.67318434,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5. Lack of Explicit Error Handling","depth":24,"bounds":{"left":0.111369684,"top":0.67278534,"width":0.094913565,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"No","depth":26,"bounds":{"left":0.12450133,"top":0.6999202,"width":0.008477394,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"try/catch","depth":27,"bounds":{"left":0.1349734,"top":0.7015164,"width":0.026928192,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Blocks:","depth":26,"bounds":{"left":0.16389628,"top":0.6999202,"width":0.019614361,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The script executes complex network operations (","depth":26,"bounds":{"left":0.18351063,"top":0.6999202,"width":0.12367021,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"loadDocuments","depth":27,"bounds":{"left":0.30917552,"top":0.7015164,"width":0.038896278,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.","depth":26,"bounds":{"left":0.12450133,"top":0.6999202,"width":0.22888963,"height":0.09377494},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":22,"bounds":{"left":0.1087101,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false}]...
|
-5936081765412384450
|
-1052937881027004452
|
idle
|
accessibility
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response...
|
65254
|
NULL
|
NULL
|
NULL
|
|
65257
|
2289
|
2
|
2026-05-20T15:36:21.201021+00:00
|
/Users/lukas/.screenpipe/data/data/2026-05-20/1779 /Users/lukas/.screenpipe/data/data/2026-05-20/1779291381201_m1.jpg...
|
Firefox
|
JY-20915 add alias for EU by LakyLak · Pull Reques JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app — Work...
|
1
|
github.com/jiminny/app/pull/12105
|
monitor_1
|
NULL
|
NULL
|
NULL
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo
Share & export
Copy
Show more options
Ask Gemini
Ask Gemini
Upload & tools
Microphone
Send message
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Summarize page
Summarize page
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
All issues(g then i)
All pull requests
All repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Pull requests (32)
Pull requests
(
32
)
Agents
Agents
Actions
Actions
Wiki
Wiki
Security and quality
Security and quality
Insights
Insights
Settings
Settings...
|
[{"role":"AXRadioButton","text [{"role":"AXRadioButton","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Close tab","depth":5,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXRadioButton","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Pipelines - jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Pipelines - jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Tab","depth":4,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Customize sidebar","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Close Google Gemini (⌃X)","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Tabs from other devices","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open history (⇧⌘H)","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open bookmarks (⌘B)","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"AI Chat settings","depth":7,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Close","depth":7,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Main menu","depth":13,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"New chat","depth":12,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Gemini","depth":15,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Chat","depth":13,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Open menu for conversation actions.","depth":13,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Conversation with Gemini","depth":15,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Conversation with Gemini","depth":16,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"You said I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.","depth":23,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Expand","depth":21,"on_screen":false,"role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a concise summary of the changes made in the code selection:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Refactoring and Dependencies","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Refactoring and Dependencies","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Replaced Interface with Implementation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Changed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusInterface","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusManager","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"in the imports and constructor dependency injection.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Redis Support:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Imported","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Illuminate\\Support\\Facades\\Redis","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to handle a newly introduced memory-logging lock mechanism.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Traits:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"WorkerIdTrait","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", along with the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"worker","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"field tracking in the final logs.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Loop and Worker Lifecycle Adjustments","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Loop and Worker Lifecycle Adjustments","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Worker Auto-Restart:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Deleted the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property, the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"constant, and the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"shouldIterateAgain()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method. The script no longer tracks execution iterations or triggers a self-managed worker restart.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Simplified Chunk Method Return:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Modified","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to always return","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Priority Handling for Throttled Entities","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Priority Handling for Throttled Entities","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Stale Record Elevation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Updated the logic in","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"rescheduleThrottledEntities()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to check if a throttled entity group is classified as \"stale\". If it is, the entities are rescheduled using","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"QueuePriorityEnum::NORMAL","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"; otherwise, they keep their respective queue priority.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Metrics, Logging, and Environment Handling","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Metrics, Logging, and Environment Handling","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Environment Tracking:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Introduced a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$environment","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property (defaulting to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"'local'","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") and a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"setEnvironment()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method to dynamically assign it.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Memory Monitoring via Datadog Gauge:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method that submits peak memory allocation to Datadog as a gauge metric (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"jiminny.reindex-memory-usage","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis Lock for Metrics Throttling:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Implemented a Redis-backed lock (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"%s-process-manager-lock","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") with a 60-second expiration inside","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to prevent over-reporting metrics.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Log Payload Update:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added total processed items (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$updatedDocuments + $deletedDocuments","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") to the final information log.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":23,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"You said are there any issues or potential bottlenecks","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"are there any issues or potential bottlenecks","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Edit","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a breakdown of the potential issues to look out for:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"1. Memory Leak Risk (OOM Crashes)","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"1. Memory Leak Risk (OOM Crashes)","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Restart Logic:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The most significant change is the removal of the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Dependency Leaks:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If any underlying service (like the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ElasticSearchClient","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", ORM, or Datadog helper) retains references to processed entities,","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Hardcoded Limits:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The code still includes","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ini_set('memory_limit', '512M');","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":". Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"2. Race Condition in Redis Lock","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"2. Race Condition in Redis Lock","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Non-Atomic Operations:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method uses three distinct steps to handle the lock:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"get()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":",","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", and","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"expire()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If multiple workers process the same","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$this->updateTarget","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"concurrently, they can both pass the","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"if (! is_null(Redis::get(...)))","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"check at the exact same millisecond. Both will then execute","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and push duplicate metrics to Datadog.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Fix:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"This should be replaced with a single atomic operation using Redis's","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"NX","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Not eXists) and","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"EX","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Expire) flags, or Laravel’s built-in atomic cache locks:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis::set($gaugeLockName, true, 'EX', 60, 'NX')","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"3. Infinite Loops and Caller Contract","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"3. Infinite Loops and Caller Contract","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Return Value Change:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"previously returned whether it should iterate again based on staleness or chunk size. It now always returns","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(unless the selection is totally empty).","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the code calling this method relies on a","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"false","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"return to pause, sleep, or gracefully terminate, changing this to an unconditional","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"could result in an aggressive \"busy loop\" that spikes CPU usage.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"4. Performance Overhead","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"4. Performance Overhead","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Expensive Garbage Collection:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Calling","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles();","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"on","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"every single chunk","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Synchronous Metric Dispatch:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::increment","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"or","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::gauge","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"5. Lack of Explicit Error Handling","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5. Lack of Explicit Error Handling","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"No","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"try/catch","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Blocks:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The script executes complex network operations (","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"loadDocuments","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":22,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":22,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Redo","depth":22,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":21,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":22,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":21,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXTextArea","text":"Ask Gemini","depth":21,"on_screen":true,"value":"Ask Gemini","help_text":"","role_description":"text entry area","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Ask Gemini","depth":22,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Upload & tools","depth":20,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Microphone","depth":20,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Send message","depth":20,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":false,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.","depth":17,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Your privacy & Gemini Opens in a new window","depth":17,"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Your privacy & Gemini","depth":18,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Opens in a new window","depth":19,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Summarize page","depth":7,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Summarize page","depth":9,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Skip to content","depth":6,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Skip to content","depth":7,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Open menu","depth":10,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"Homepage (g then d)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"jiminny","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"jiminny","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"app","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"app","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Search or jump to…","depth":9,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Type","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"/","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to search","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Chat with Copilot","depth":10,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXMenuButton","text":"Open Copilot…","depth":9,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXMenuButton","text":"Create new...","depth":9,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"All issues(g then i)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"All pull requests","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"All repositories","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"You have unread notifications(g then n)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Open user navigation menu","depth":9,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Repository navigation","depth":9,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Repository navigation","depth":10,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Code","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Code","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Pull requests (32)","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Pull requests","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"32","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Agents","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Agents","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Actions","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Actions","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Wiki","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Wiki","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Security and quality","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Security and quality","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Insights","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Insights","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Settings","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Settings","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"}]...
|
-6845205575762006669
|
-1052936747159816420
|
idle
|
accessibility
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo
Share & export
Copy
Show more options
Ask Gemini
Ask Gemini
Upload & tools
Microphone
Send message
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Summarize page
Summarize page
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
All issues(g then i)
All pull requests
All repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Pull requests (32)
Pull requests
(
32
)
Agents
Agents
Actions
Actions
Wiki
Wiki
Security and quality
Security and quality
Insights
Insights
Settings
Settings...
|
65255
|
NULL
|
NULL
|
NULL
|
|
65258
|
2290
|
3
|
2026-05-20T15:36:33.871971+00:00
|
/Users/lukas/.screenpipe/data/data/2026-05-20/1779 /Users/lukas/.screenpipe/data/data/2026-05-20/1779291393871_m2.jpg...
|
Firefox
|
JY-20915 add alias for EU by LakyLak · Pull Reques JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app — Work...
|
1
|
github.com/jiminny/app/pull/12105
|
monitor_2
|
NULL
|
NULL
|
NULL
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo
Share & export
Copy
Show more options
Ask Gemini
Ask Gemini
Upload & tools
Microphone
Send message
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Summarize page
Summarize page
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
All issues(g then i)
All pull requests
All repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Pull requests (32)
Pull requests
(
32
)
Agents
Agents
Actions
Actions
Wiki
Wiki
Security and quality
Security and quality
Insights
Insights
Settings
Settings
JY-20915 add alias for EU #12105 Edit title
JY-20915 add alias for EU
#
12105
Edit title
Awaiting approval
Awaiting approval
Code
Code
Open
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Lines changed: 365 additions & 1 deletion
Conversation (1)
Conversation
(
1
)
Commits (5)
Commits
(
5
)
Checks (3)
Checks
(
3
)
Files changed (4)
Files changed
(
4
)
Open
JY-20915 add alias for EU #12105 LakyLak wants to merge 5 commits into master from JY-20915-add-domain-specific-email-text-relay Copy head branch name to clipboard
JY-20915 add alias for EU...
|
[{"role":"AXRadioButton","text [{"role":"AXRadioButton","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":4,"bounds":{"left":0.0,"top":0.0518755,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.06304868,"width":0.10106383,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":4,"bounds":{"left":0.0,"top":0.08459697,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.09577015,"width":0.10721409,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.11731844,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.12849163,"width":0.15791224,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.15003991,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.16121309,"width":0.1796875,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.18276137,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.19393456,"width":0.12699468,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Close tab","depth":5,"bounds":{"left":0.06732048,"top":0.18994413,"width":0.007978723,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXRadioButton","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":4,"bounds":{"left":0.0,"top":0.21548285,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":5,"bounds":{"left":0.013297873,"top":0.22665602,"width":0.10139628,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":4,"bounds":{"left":0.0,"top":0.2482043,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.25937748,"width":0.18068483,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":4,"bounds":{"left":0.0,"top":0.28092578,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":5,"bounds":{"left":0.013297873,"top":0.29209897,"width":0.4644282,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Pipelines - jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.31364724,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Pipelines - jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.32482043,"width":0.039228722,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.3463687,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.3575419,"width":0.15824468,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Tab","depth":4,"bounds":{"left":0.0028257978,"top":0.38068634,"width":0.07413564,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Customize sidebar","depth":6,"bounds":{"left":0.0028257978,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Close Google Gemini (⌃X)","depth":6,"bounds":{"left":0.013796543,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Tabs from other devices","depth":6,"bounds":{"left":0.024933511,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open history (⇧⌘H)","depth":6,"bounds":{"left":0.036070477,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open bookmarks (⌘B)","depth":6,"bounds":{"left":0.04720745,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"AI Chat settings","depth":7,"bounds":{"left":0.35854387,"top":0.055067837,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Close","depth":7,"bounds":{"left":0.37051198,"top":0.055067837,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Main menu","depth":13,"bounds":{"left":0.08228058,"top":0.09736632,"width":0.015957447,"height":0.03830806},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"New chat","depth":12,"bounds":{"left":0.09823803,"top":0.10215483,"width":0.028091755,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Gemini","depth":15,"bounds":{"left":0.1008976,"top":0.10654429,"width":0.02144282,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Chat","depth":13,"bounds":{"left":0.35854387,"top":0.10215483,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Open menu for conversation actions.","depth":13,"bounds":{"left":0.37051198,"top":0.10215483,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Conversation with Gemini","depth":15,"bounds":{"left":0.079288565,"top":0.14445332,"width":0.0003324468,"height":0.0007980846},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Conversation with Gemini","depth":16,"bounds":{"left":0.079288565,"top":0.14684756,"width":0.1200133,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"You said I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.","depth":23,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Expand","depth":21,"on_screen":false,"role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a concise summary of the changes made in the code selection:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Refactoring and Dependencies","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Refactoring and Dependencies","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Replaced Interface with Implementation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Changed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusInterface","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusManager","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"in the imports and constructor dependency injection.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Redis Support:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Imported","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Illuminate\\Support\\Facades\\Redis","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to handle a newly introduced memory-logging lock mechanism.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Traits:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"WorkerIdTrait","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", along with the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"worker","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"field tracking in the final logs.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Loop and Worker Lifecycle Adjustments","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Loop and Worker Lifecycle Adjustments","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Worker Auto-Restart:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Deleted the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property, the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"constant, and the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"shouldIterateAgain()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method. The script no longer tracks execution iterations or triggers a self-managed worker restart.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Simplified Chunk Method Return:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Modified","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to always return","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Priority Handling for Throttled Entities","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Priority Handling for Throttled Entities","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Stale Record Elevation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Updated the logic in","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"rescheduleThrottledEntities()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to check if a throttled entity group is classified as \"stale\". If it is, the entities are rescheduled using","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"QueuePriorityEnum::NORMAL","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"; otherwise, they keep their respective queue priority.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Metrics, Logging, and Environment Handling","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Metrics, Logging, and Environment Handling","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Environment Tracking:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Introduced a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$environment","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property (defaulting to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"'local'","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") and a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"setEnvironment()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method to dynamically assign it.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Memory Monitoring via Datadog Gauge:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method that submits peak memory allocation to Datadog as a gauge metric (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"jiminny.reindex-memory-usage","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis Lock for Metrics Throttling:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Implemented a Redis-backed lock (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"%s-process-manager-lock","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") with a 60-second expiration inside","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to prevent over-reporting metrics.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Log Payload Update:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added total processed items (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$updatedDocuments + $deletedDocuments","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") to the final information log.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":23,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"You said are there any issues or potential bottlenecks","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"are there any issues or potential bottlenecks","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Edit","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a breakdown of the potential issues to look out for:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"1. Memory Leak Risk (OOM Crashes)","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"1. Memory Leak Risk (OOM Crashes)","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Restart Logic:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The most significant change is the removal of the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Dependency Leaks:","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.050033245,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If any underlying service (like the","depth":26,"bounds":{"left":0.17453457,"top":0.0,"width":0.08277926,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ElasticSearchClient","depth":27,"bounds":{"left":0.25930852,"top":0.0,"width":0.056848403,"height":0.015163607},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", ORM, or Datadog helper) retains references to processed entities,","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.21775267,"height":0.03631285},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"bounds":{"left":0.26811835,"top":0.0,"width":0.056848403,"height":0.015163607},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.23138298,"height":0.05546688},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Hardcoded Limits:","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.047041222,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The code still includes","depth":26,"bounds":{"left":0.17154256,"top":0.0,"width":0.05651596,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ini_set('memory_limit', '512M');","depth":27,"bounds":{"left":0.23005319,"top":0.0,"width":0.095744684,"height":0.015163607},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":". Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.23055187,"height":0.05546688},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"2. Race Condition in Redis Lock","depth":23,"bounds":{"left":0.111369684,"top":0.07940942,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"2. Race Condition in Redis Lock","depth":24,"bounds":{"left":0.111369684,"top":0.079010375,"width":0.09125665,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Non-Atomic Operations:","depth":26,"bounds":{"left":0.12450133,"top":0.10614525,"width":0.062333778,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The","depth":26,"bounds":{"left":0.18683511,"top":0.10614525,"width":0.011635638,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage","depth":27,"bounds":{"left":0.20046543,"top":0.10774142,"width":0.041888297,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method uses three distinct steps to handle the lock:","depth":26,"bounds":{"left":0.12450133,"top":0.10614525,"width":0.22722739,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"get()","depth":27,"bounds":{"left":0.14860372,"top":0.12689546,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":",","depth":26,"bounds":{"left":0.16555852,"top":0.12529927,"width":0.002493351,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"bounds":{"left":0.17004654,"top":0.12689546,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", and","depth":26,"bounds":{"left":0.18700133,"top":0.12529927,"width":0.013131649,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"expire()","depth":27,"bounds":{"left":0.20212767,"top":0.12689546,"width":0.023936171,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"bounds":{"left":0.22805852,"top":0.12529927,"width":0.0013297872,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"bounds":{"left":0.12450133,"top":0.15403032,"width":0.040059842,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If multiple workers process the same","depth":26,"bounds":{"left":0.16456117,"top":0.15403032,"width":0.09275266,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$this->updateTarget","depth":27,"bounds":{"left":0.25930852,"top":0.15562649,"width":0.056848403,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"concurrently, they can both pass the","depth":26,"bounds":{"left":0.12450133,"top":0.15403032,"width":0.22855718,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"if (! is_null(Redis::get(...)))","depth":27,"bounds":{"left":0.18351063,"top":0.17478053,"width":0.09275266,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"check at the exact same millisecond. Both will then execute","depth":26,"bounds":{"left":0.12450133,"top":0.17318435,"width":0.21575798,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"bounds":{"left":0.2122673,"top":0.19393456,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and push duplicate metrics to Datadog.","depth":26,"bounds":{"left":0.22922207,"top":0.19233839,"width":0.09823803,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Fix:","depth":26,"bounds":{"left":0.12450133,"top":0.22106944,"width":0.019281914,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"This should be replaced with a single atomic operation using Redis's","depth":26,"bounds":{"left":0.14378324,"top":0.22106944,"width":0.16821809,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"NX","depth":27,"bounds":{"left":0.31399602,"top":0.22266561,"width":0.005984043,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Not eXists) and","depth":26,"bounds":{"left":0.12450133,"top":0.22106944,"width":0.22855718,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"EX","depth":27,"bounds":{"left":0.13696809,"top":0.24181964,"width":0.005984043,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Expire) flags, or Laravel’s built-in atomic cache locks:","depth":26,"bounds":{"left":0.14494681,"top":0.24022347,"width":0.13264628,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis::set($gaugeLockName, true, 'EX', 60, 'NX')","depth":27,"bounds":{"left":0.12649602,"top":0.26097366,"width":0.14361702,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"bounds":{"left":0.27210772,"top":0.25937748,"width":0.0013297872,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"3. Infinite Loops and Caller Contract","depth":23,"bounds":{"left":0.111369684,"top":0.30925778,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"3. Infinite Loops and Caller Contract","depth":24,"bounds":{"left":0.111369684,"top":0.30885875,"width":0.10571808,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Return Value Change:","depth":26,"bounds":{"left":0.12450133,"top":0.33599362,"width":0.055352394,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"bounds":{"left":0.18301196,"top":0.33758977,"width":0.06881649,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"previously returned whether it should iterate again based on staleness or chunk size. It now always returns","depth":26,"bounds":{"left":0.12450133,"top":0.33599362,"width":0.2237367,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"bounds":{"left":0.29438165,"top":0.3567438,"width":0.011968086,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(unless the selection is totally empty).","depth":26,"bounds":{"left":0.12450133,"top":0.35514766,"width":0.2130984,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"bounds":{"left":0.12450133,"top":0.40303272,"width":0.040059842,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the code calling this method relies on a","depth":26,"bounds":{"left":0.16456117,"top":0.40303272,"width":0.10305851,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"false","depth":27,"bounds":{"left":0.26961437,"top":0.4046289,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"return to pause, sleep, or gracefully terminate, changing this to an unconditional","depth":26,"bounds":{"left":0.12450133,"top":0.40303272,"width":0.22589761,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"bounds":{"left":0.26163563,"top":0.4237829,"width":0.011968086,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"could result in an aggressive \"busy loop\" that spikes CPU usage.","depth":26,"bounds":{"left":0.12450133,"top":0.42218676,"width":0.22290559,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"4. Performance Overhead","depth":23,"bounds":{"left":0.111369684,"top":0.49122107,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"4. Performance Overhead","depth":24,"bounds":{"left":0.111369684,"top":0.49082202,"width":0.076130316,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Expensive Garbage Collection:","depth":26,"bounds":{"left":0.12450133,"top":0.5179569,"width":0.078457445,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Calling","depth":26,"bounds":{"left":0.20295878,"top":0.5179569,"width":0.019281914,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles();","depth":27,"bounds":{"left":0.22423537,"top":0.51955307,"width":0.059840426,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"on","depth":26,"bounds":{"left":0.28607047,"top":0.5179569,"width":0.00880984,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"every single chunk","depth":26,"bounds":{"left":0.29488033,"top":0.5179569,"width":0.045545213,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.","depth":26,"bounds":{"left":0.12450133,"top":0.5179569,"width":0.23420878,"height":0.05546688},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Synchronous Metric Dispatch:","depth":26,"bounds":{"left":0.12450133,"top":0.584996,"width":0.07712766,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the","depth":26,"bounds":{"left":0.20162898,"top":0.584996,"width":0.015292553,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::increment","depth":27,"bounds":{"left":0.21891622,"top":0.5865922,"width":0.053856384,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"or","depth":26,"bounds":{"left":0.27476728,"top":0.584996,"width":0.0078125,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::gauge","depth":27,"bounds":{"left":0.28457448,"top":0.5865922,"width":0.041888297,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.","depth":26,"bounds":{"left":0.12450133,"top":0.584996,"width":0.23420878,"height":0.05546688},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"5. Lack of Explicit Error Handling","depth":23,"bounds":{"left":0.111369684,"top":0.67318434,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5. Lack of Explicit Error Handling","depth":24,"bounds":{"left":0.111369684,"top":0.67278534,"width":0.094913565,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"No","depth":26,"bounds":{"left":0.12450133,"top":0.6999202,"width":0.008477394,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"try/catch","depth":27,"bounds":{"left":0.1349734,"top":0.7015164,"width":0.026928192,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Blocks:","depth":26,"bounds":{"left":0.16389628,"top":0.6999202,"width":0.019614361,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The script executes complex network operations (","depth":26,"bounds":{"left":0.18351063,"top":0.6999202,"width":0.12367021,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"loadDocuments","depth":27,"bounds":{"left":0.30917552,"top":0.7015164,"width":0.038896278,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.","depth":26,"bounds":{"left":0.12450133,"top":0.6999202,"width":0.22888963,"height":0.09377494},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":22,"bounds":{"left":0.1087101,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":22,"bounds":{"left":0.124667555,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Redo","depth":22,"bounds":{"left":0.140625,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":21,"bounds":{"left":0.15658244,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":22,"bounds":{"left":0.17253989,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":21,"bounds":{"left":0.18849733,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXTextArea","text":"Ask Gemini","depth":21,"bounds":{"left":0.14594415,"top":0.86632085,"width":0.17021276,"height":0.01915403},"on_screen":true,"value":"Ask Gemini","help_text":"","role_description":"text entry area","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Ask Gemini","depth":22,"bounds":{"left":0.14594415,"top":0.8667199,"width":0.030418882,"height":0.018355945},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Upload & tools","depth":20,"bounds":{"left":0.12732713,"top":0.8567438,"width":0.015957447,"height":0.03830806},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Microphone","depth":20,"bounds":{"left":0.32413563,"top":0.86153233,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Send message","depth":20,"bounds":{"left":0.33211437,"top":0.8631285,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":false,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.","depth":17,"bounds":{"left":0.102726065,"top":0.9173983,"width":0.21775267,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Your privacy & Gemini Opens in a new window","depth":17,"bounds":{"left":0.32047874,"top":0.9173983,"width":0.043882977,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Your privacy & Gemini","depth":18,"bounds":{"left":0.32047874,"top":0.9173983,"width":0.043882977,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Opens in a new window","depth":19,"bounds":{"left":0.079288565,"top":0.91660017,"width":0.04720745,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Summarize page","depth":7,"bounds":{"left":0.08494016,"top":0.95730245,"width":0.053523935,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Summarize page","depth":9,"bounds":{"left":0.09059176,"top":0.96249,"width":0.042220745,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Skip to content","depth":6,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Skip to content","depth":7,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Open menu","depth":10,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"Homepage (g then d)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"jiminny","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"jiminny","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"app","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"app","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Search or jump to…","depth":9,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Type","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"/","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to search","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Chat with Copilot","depth":10,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXMenuButton","text":"Open Copilot…","depth":9,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXMenuButton","text":"Create new...","depth":9,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"All issues(g then i)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"All pull requests","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"All repositories","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"You have unread notifications(g then n)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Open user navigation menu","depth":9,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Repository navigation","depth":9,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Repository navigation","depth":10,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Code","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Code","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Pull requests (32)","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Pull requests","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"32","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Agents","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Agents","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Actions","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Actions","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Wiki","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Wiki","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Security and quality","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Security and quality","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Insights","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Insights","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Settings","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Settings","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"JY-20915 add alias for EU #12105 Edit title","depth":13,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"JY-20915 add alias for EU","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"#","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"12105","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Edit title","depth":14,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Awaiting approval","depth":13,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Awaiting approval","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXMenuButton","text":"Code","depth":13,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXStaticText","text":"Code","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Open","depth":13,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"LakyLak","depth":15,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"wants to merge 5 commits into","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"master","depth":15,"on_screen":false,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"master","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"from","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915-add-domain-specific-email-text-relay","depth":16,"on_screen":false,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915-add-domain-specific-email-text-relay","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy head branch name to clipboard","depth":16,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Lines changed: 365 additions & 1 deletion","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Conversation (1)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"Conversation","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"1","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Commits (5)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Commits","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Checks (3)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Checks","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"3","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Files changed (4)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Files changed","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"4","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Open","depth":14,"bounds":{"left":0.50332445,"top":0.07222666,"width":0.011968086,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"JY-20915 add alias for EU #12105 LakyLak wants to merge 5 commits into master from JY-20915-add-domain-specific-email-text-relay Copy head branch name to clipboard","depth":14,"bounds":{"left":0.5219415,"top":0.058260176,"width":0.22922207,"height":0.042298485},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915 add alias for EU","depth":16,"bounds":{"left":0.5219415,"top":0.05865922,"width":0.05817819,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false}]...
|
-4648784576994277641
|
-1052796009672575204
|
idle
|
accessibility
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo
Share & export
Copy
Show more options
Ask Gemini
Ask Gemini
Upload & tools
Microphone
Send message
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Summarize page
Summarize page
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
All issues(g then i)
All pull requests
All repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Pull requests (32)
Pull requests
(
32
)
Agents
Agents
Actions
Actions
Wiki
Wiki
Security and quality
Security and quality
Insights
Insights
Settings
Settings
JY-20915 add alias for EU #12105 Edit title
JY-20915 add alias for EU
#
12105
Edit title
Awaiting approval
Awaiting approval
Code
Code
Open
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Lines changed: 365 additions & 1 deletion
Conversation (1)
Conversation
(
1
)
Commits (5)
Commits
(
5
)
Checks (3)
Checks
(
3
)
Files changed (4)
Files changed
(
4
)
Open
JY-20915 add alias for EU #12105 LakyLak wants to merge 5 commits into master from JY-20915-add-domain-specific-email-text-relay Copy head branch name to clipboard
JY-20915 add alias for EU...
|
NULL
|
NULL
|
NULL
|
NULL
|
|
65259
|
2289
|
3
|
2026-05-20T15:36:51.589142+00:00
|
/Users/lukas/.screenpipe/data/data/2026-05-20/1779 /Users/lukas/.screenpipe/data/data/2026-05-20/1779291411589_m1.jpg...
|
Firefox
|
JY-20915 add alias for EU by LakyLak · Pull Reques JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app — Work...
|
1
|
github.com/jiminny/app/pull/12105
|
monitor_1
|
NULL
|
NULL
|
NULL
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo
Share & export
Copy
Show more options
Ask Gemini
Ask Gemini
Upload & tools
Microphone
Send message
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Summarize page
Summarize page
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
All issues(g then i)
All pull requests
All repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Pull requests (32)
Pull requests
(
32
)
Agents
Agents
Actions
Actions
Wiki
Wiki
Security and quality
Security and quality
Insights
Insights
Settings
Settings
JY-20915 add alias for EU #12105 Edit title
JY-20915 add alias for EU
#
12105
Edit title
Awaiting approval
Awaiting approval
Code
Code
Open
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Lines changed: 365 additions & 1 deletion
Conversation (1)
Conversation
(
1
)
Commits (5)
Commits
(
5
)
Checks (3)
Checks
(
3
)
Files changed (4)
Files changed
(
4
)
Open
JY-20915 add alias for EU #12105 LakyLak wants to merge 5 commits into master from JY-20915-add-domain-specific-email-text-relay Copy head branch name to clipboard
JY-20915 add alias for EU
JY-20915 add alias for EU
#
12105
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Conversation
Conversation
@LakyLak
Show options
LakyLak commented yesterday
LakyLak
LakyLak
commented
yesterday
yesterday
JIRA: JY-20915
JIRA:
JY-20915
JY-20915
Changes:
Changes:
Support text relay for EU
Prevent errors from EU on US
Add or remove reactions
LakyLak
LakyLak
added
5
commits
yesterday
yesterday
@LakyLak
JY-20915
JY-20915
add alias for EU
add alias for EU
8 / 10 checks OK
e5255d5
e5255d5
@LakyLak
JY-20915
JY-20915
fix env recognization
fix env recognization
11 / 12 checks OK
8953054
8953054
@LakyLak
JY-20915
JY-20915
code smells & tests
code smells & tests
11 / 12 checks OK
73d4bea
73d4bea
@LakyLak
JY-20915
JY-20915
add more tests
add more tests
11 / 12 checks OK
5d2dd3e
5d2dd3e
@LakyLak
JY-20915
JY-20915
add sms received test
add sms received test
12 / 12 checks OK...
|
[{"role":"AXRadioButton","text [{"role":"AXRadioButton","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Close tab","depth":5,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXRadioButton","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Pipelines - jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Pipelines - jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Tab","depth":4,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Customize sidebar","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Close Google Gemini (⌃X)","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Tabs from other devices","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open history (⇧⌘H)","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open bookmarks (⌘B)","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"AI Chat settings","depth":7,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Close","depth":7,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Main menu","depth":13,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"New chat","depth":12,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Gemini","depth":15,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Chat","depth":13,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Open menu for conversation actions.","depth":13,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Conversation with Gemini","depth":15,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Conversation with Gemini","depth":16,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"You said I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.","depth":23,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Expand","depth":21,"on_screen":false,"role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a concise summary of the changes made in the code selection:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Refactoring and Dependencies","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Refactoring and Dependencies","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Replaced Interface with Implementation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Changed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusInterface","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusManager","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"in the imports and constructor dependency injection.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Redis Support:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Imported","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Illuminate\\Support\\Facades\\Redis","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to handle a newly introduced memory-logging lock mechanism.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Traits:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"WorkerIdTrait","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", along with the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"worker","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"field tracking in the final logs.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Loop and Worker Lifecycle Adjustments","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Loop and Worker Lifecycle Adjustments","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Worker Auto-Restart:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Deleted the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property, the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"constant, and the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"shouldIterateAgain()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method. The script no longer tracks execution iterations or triggers a self-managed worker restart.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Simplified Chunk Method Return:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Modified","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to always return","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Priority Handling for Throttled Entities","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Priority Handling for Throttled Entities","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Stale Record Elevation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Updated the logic in","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"rescheduleThrottledEntities()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to check if a throttled entity group is classified as \"stale\". If it is, the entities are rescheduled using","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"QueuePriorityEnum::NORMAL","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"; otherwise, they keep their respective queue priority.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Metrics, Logging, and Environment Handling","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Metrics, Logging, and Environment Handling","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Environment Tracking:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Introduced a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$environment","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property (defaulting to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"'local'","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") and a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"setEnvironment()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method to dynamically assign it.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Memory Monitoring via Datadog Gauge:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method that submits peak memory allocation to Datadog as a gauge metric (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"jiminny.reindex-memory-usage","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis Lock for Metrics Throttling:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Implemented a Redis-backed lock (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"%s-process-manager-lock","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") with a 60-second expiration inside","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to prevent over-reporting metrics.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Log Payload Update:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added total processed items (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$updatedDocuments + $deletedDocuments","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") to the final information log.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":23,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"You said are there any issues or potential bottlenecks","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"are there any issues or potential bottlenecks","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Edit","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a breakdown of the potential issues to look out for:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"1. Memory Leak Risk (OOM Crashes)","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"1. Memory Leak Risk (OOM Crashes)","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Restart Logic:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The most significant change is the removal of the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Dependency Leaks:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If any underlying service (like the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ElasticSearchClient","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", ORM, or Datadog helper) retains references to processed entities,","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Hardcoded Limits:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The code still includes","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ini_set('memory_limit', '512M');","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":". Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"2. Race Condition in Redis Lock","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"2. Race Condition in Redis Lock","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Non-Atomic Operations:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method uses three distinct steps to handle the lock:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"get()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":",","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", and","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"expire()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If multiple workers process the same","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$this->updateTarget","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"concurrently, they can both pass the","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"if (! is_null(Redis::get(...)))","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"check at the exact same millisecond. Both will then execute","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and push duplicate metrics to Datadog.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Fix:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"This should be replaced with a single atomic operation using Redis's","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"NX","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Not eXists) and","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"EX","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Expire) flags, or Laravel’s built-in atomic cache locks:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis::set($gaugeLockName, true, 'EX', 60, 'NX')","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"3. Infinite Loops and Caller Contract","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"3. Infinite Loops and Caller Contract","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Return Value Change:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"previously returned whether it should iterate again based on staleness or chunk size. It now always returns","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(unless the selection is totally empty).","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the code calling this method relies on a","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"false","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"return to pause, sleep, or gracefully terminate, changing this to an unconditional","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"could result in an aggressive \"busy loop\" that spikes CPU usage.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"4. Performance Overhead","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"4. Performance Overhead","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Expensive Garbage Collection:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Calling","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles();","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"on","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"every single chunk","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Synchronous Metric Dispatch:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::increment","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"or","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::gauge","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"5. Lack of Explicit Error Handling","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5. Lack of Explicit Error Handling","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"No","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"try/catch","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Blocks:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The script executes complex network operations (","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"loadDocuments","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":22,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":22,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Redo","depth":22,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":21,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":22,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":21,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXTextArea","text":"Ask Gemini","depth":21,"on_screen":true,"value":"Ask Gemini","help_text":"","role_description":"text entry area","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Ask Gemini","depth":22,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Upload & tools","depth":20,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Microphone","depth":20,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Send message","depth":20,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":false,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.","depth":17,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Your privacy & Gemini Opens in a new window","depth":17,"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Your privacy & Gemini","depth":18,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Opens in a new window","depth":19,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Summarize page","depth":7,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Summarize page","depth":9,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Skip to content","depth":6,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Skip to content","depth":7,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Open menu","depth":10,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"Homepage (g then d)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"jiminny","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"jiminny","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"app","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"app","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Search or jump to…","depth":9,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Type","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"/","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to search","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Chat with Copilot","depth":10,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXMenuButton","text":"Open Copilot…","depth":9,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXMenuButton","text":"Create new...","depth":9,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"All issues(g then i)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"All pull requests","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"All repositories","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"You have unread notifications(g then n)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Open user navigation menu","depth":9,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Repository navigation","depth":9,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Repository navigation","depth":10,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Code","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Code","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Pull requests (32)","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Pull requests","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"32","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Agents","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Agents","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Actions","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Actions","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Wiki","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Wiki","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Security and quality","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Security and quality","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Insights","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Insights","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Settings","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Settings","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"JY-20915 add alias for EU #12105 Edit title","depth":13,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"JY-20915 add alias for EU","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"#","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"12105","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Edit title","depth":14,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Awaiting approval","depth":13,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Awaiting approval","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXMenuButton","text":"Code","depth":13,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXStaticText","text":"Code","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Open","depth":13,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"LakyLak","depth":15,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"wants to merge 5 commits into","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"master","depth":15,"on_screen":false,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"master","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"from","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915-add-domain-specific-email-text-relay","depth":16,"on_screen":false,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915-add-domain-specific-email-text-relay","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy head branch name to clipboard","depth":16,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Lines changed: 365 additions & 1 deletion","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Conversation (1)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"Conversation","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"1","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Commits (5)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Commits","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Checks (3)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Checks","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"3","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Files changed (4)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Files changed","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"4","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Open","depth":14,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"JY-20915 add alias for EU #12105 LakyLak wants to merge 5 commits into master from JY-20915-add-domain-specific-email-text-relay Copy head branch name to clipboard","depth":14,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915 add alias for EU","depth":16,"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915 add alias for EU","depth":17,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"#","depth":16,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"12105","depth":16,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"LakyLak","depth":18,"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":19,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"wants to merge 5 commits into","depth":18,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"master","depth":18,"on_screen":true,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"master","depth":19,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"from","depth":19,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915-add-domain-specific-email-text-relay","depth":19,"on_screen":true,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915-add-domain-specific-email-text-relay","depth":20,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy head branch name to clipboard","depth":19,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Conversation","depth":12,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Conversation","depth":13,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@LakyLak","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show options","depth":15,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXHeading","text":"LakyLak commented yesterday","depth":14,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXLink","text":"LakyLak","depth":16,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"commented","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"yesterday","depth":15,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"yesterday","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"JIRA: JY-20915","depth":16,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"JIRA:","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915","depth":17,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Changes:","depth":16,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Changes:","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Support text relay for EU","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Prevent errors from EU on US","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Add or remove reactions","depth":16,"on_screen":false,"help_text":"","role_description":"summary","subrole":"AXSummary","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"LakyLak","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"added","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"commits","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"yesterday","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"yesterday","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@LakyLak","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"JY-20915","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"add alias for EU","depth":14,"on_screen":false,"help_text":"JY-20915 add alias for EU","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"add alias for EU","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"8 / 10 checks OK","depth":14,"on_screen":false,"help_text":"","role_description":"summary","subrole":"AXSummary","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"e5255d5","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"e5255d5","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@LakyLak","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"JY-20915","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"fix env recognization","depth":14,"on_screen":false,"help_text":"JY-20915 fix env recognization","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"fix env recognization","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"11 / 12 checks OK","depth":14,"on_screen":false,"help_text":"","role_description":"summary","subrole":"AXSummary","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"8953054","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"8953054","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@LakyLak","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"JY-20915","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"code smells & tests","depth":14,"on_screen":false,"help_text":"JY-20915 code smells & tests","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"code smells & tests","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"11 / 12 checks OK","depth":14,"on_screen":false,"help_text":"","role_description":"summary","subrole":"AXSummary","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"73d4bea","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"73d4bea","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@LakyLak","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"JY-20915","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"add more tests","depth":14,"on_screen":false,"help_text":"JY-20915 add more tests","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"add more tests","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"11 / 12 checks OK","depth":14,"on_screen":false,"help_text":"","role_description":"summary","subrole":"AXSummary","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"5d2dd3e","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"5d2dd3e","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@LakyLak","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"JY-20915","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"add sms received test","depth":14,"on_screen":false,"help_text":"JY-20915 add sms received test","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"add sms received test","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"12 / 12 checks OK","depth":14,"on_screen":false,"help_text":"","role_description":"summary","subrole":"AXSummary","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false}]...
|
380400995060395706
|
-1066448645554686184
|
idle
|
accessibility
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo
Share & export
Copy
Show more options
Ask Gemini
Ask Gemini
Upload & tools
Microphone
Send message
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Summarize page
Summarize page
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
All issues(g then i)
All pull requests
All repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Pull requests (32)
Pull requests
(
32
)
Agents
Agents
Actions
Actions
Wiki
Wiki
Security and quality
Security and quality
Insights
Insights
Settings
Settings
JY-20915 add alias for EU #12105 Edit title
JY-20915 add alias for EU
#
12105
Edit title
Awaiting approval
Awaiting approval
Code
Code
Open
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Lines changed: 365 additions & 1 deletion
Conversation (1)
Conversation
(
1
)
Commits (5)
Commits
(
5
)
Checks (3)
Checks
(
3
)
Files changed (4)
Files changed
(
4
)
Open
JY-20915 add alias for EU #12105 LakyLak wants to merge 5 commits into master from JY-20915-add-domain-specific-email-text-relay Copy head branch name to clipboard
JY-20915 add alias for EU
JY-20915 add alias for EU
#
12105
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Conversation
Conversation
@LakyLak
Show options
LakyLak commented yesterday
LakyLak
LakyLak
commented
yesterday
yesterday
JIRA: JY-20915
JIRA:
JY-20915
JY-20915
Changes:
Changes:
Support text relay for EU
Prevent errors from EU on US
Add or remove reactions
LakyLak
LakyLak
added
5
commits
yesterday
yesterday
@LakyLak
JY-20915
JY-20915
add alias for EU
add alias for EU
8 / 10 checks OK
e5255d5
e5255d5
@LakyLak
JY-20915
JY-20915
fix env recognization
fix env recognization
11 / 12 checks OK
8953054
8953054
@LakyLak
JY-20915
JY-20915
code smells & tests
code smells & tests
11 / 12 checks OK
73d4bea
73d4bea
@LakyLak
JY-20915
JY-20915
add more tests
add more tests
11 / 12 checks OK
5d2dd3e
5d2dd3e
@LakyLak
JY-20915
JY-20915
add sms received test
add sms received test
12 / 12 checks OK...
|
NULL
|
NULL
|
NULL
|
NULL
|
|
65260
|
2290
|
4
|
2026-05-20T15:37:04.291496+00:00
|
/Users/lukas/.screenpipe/data/data/2026-05-20/1779 /Users/lukas/.screenpipe/data/data/2026-05-20/1779291424291_m2.jpg...
|
Firefox
|
JY-20915 add alias for EU by LakyLak · Pull Reques JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app — Work...
|
1
|
github.com/jiminny/app/pull/12105
|
monitor_2
|
NULL
|
NULL
|
NULL
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo
Share & export
Copy
Show more options
Ask Gemini
Ask Gemini
Upload & tools
Microphone
Send message
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Summarize page
Summarize page
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
All issues(g then i)
All pull requests
All repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Pull requests (32)
Pull requests
(
32
)
Agents
Agents
Actions
Actions
Wiki
Wiki
Security and quality
Security and quality
Insights
Insights
Settings
Settings
JY-20915 add alias for EU #12105 Edit title
JY-20915 add alias for EU
#
12105
Edit title
Awaiting approval
Awaiting approval
Code
Code
Open
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Lines changed: 365 additions & 1 deletion
Conversation (1)
Conversation
(
1
)
Commits (5)
Commits
(
5
)
Checks (3)
Checks
(
3
)
Files changed (4)
Files changed
(
4
)
Open
JY-20915 add alias for EU #12105 LakyLak wants to merge 5 commits into master from JY-20915-add-domain-specific-email-text-relay Copy head branch name to clipboard
JY-20915 add alias for EU
JY-20915 add alias for EU...
|
[{"role":"AXRadioButton","text [{"role":"AXRadioButton","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":4,"bounds":{"left":0.0,"top":0.0518755,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.06304868,"width":0.10106383,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":4,"bounds":{"left":0.0,"top":0.08459697,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.09577015,"width":0.10721409,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.11731844,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.12849163,"width":0.15791224,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.15003991,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.16121309,"width":0.1796875,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.18276137,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.19393456,"width":0.12699468,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Close tab","depth":5,"bounds":{"left":0.06732048,"top":0.18994413,"width":0.007978723,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXRadioButton","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":4,"bounds":{"left":0.0,"top":0.21548285,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":5,"bounds":{"left":0.013297873,"top":0.22665602,"width":0.10139628,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":4,"bounds":{"left":0.0,"top":0.2482043,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.25937748,"width":0.18068483,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":4,"bounds":{"left":0.0,"top":0.28092578,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":5,"bounds":{"left":0.013297873,"top":0.29209897,"width":0.4644282,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Pipelines - jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.31364724,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Pipelines - jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.32482043,"width":0.039228722,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.3463687,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.3575419,"width":0.15824468,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Tab","depth":4,"bounds":{"left":0.0028257978,"top":0.38068634,"width":0.07413564,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Customize sidebar","depth":6,"bounds":{"left":0.0028257978,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Close Google Gemini (⌃X)","depth":6,"bounds":{"left":0.013796543,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Tabs from other devices","depth":6,"bounds":{"left":0.024933511,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open history (⇧⌘H)","depth":6,"bounds":{"left":0.036070477,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open bookmarks (⌘B)","depth":6,"bounds":{"left":0.04720745,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"AI Chat settings","depth":7,"bounds":{"left":0.35854387,"top":0.055067837,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Close","depth":7,"bounds":{"left":0.37051198,"top":0.055067837,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Main menu","depth":13,"bounds":{"left":0.08228058,"top":0.09736632,"width":0.015957447,"height":0.03830806},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"New chat","depth":12,"bounds":{"left":0.09823803,"top":0.10215483,"width":0.028091755,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Gemini","depth":15,"bounds":{"left":0.1008976,"top":0.10654429,"width":0.02144282,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Chat","depth":13,"bounds":{"left":0.35854387,"top":0.10215483,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Open menu for conversation actions.","depth":13,"bounds":{"left":0.37051198,"top":0.10215483,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Conversation with Gemini","depth":15,"bounds":{"left":0.079288565,"top":0.14445332,"width":0.0003324468,"height":0.0007980846},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Conversation with Gemini","depth":16,"bounds":{"left":0.079288565,"top":0.14684756,"width":0.1200133,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"You said I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.","depth":23,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Expand","depth":21,"on_screen":false,"role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a concise summary of the changes made in the code selection:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Refactoring and Dependencies","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Refactoring and Dependencies","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Replaced Interface with Implementation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Changed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusInterface","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusManager","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"in the imports and constructor dependency injection.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Redis Support:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Imported","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Illuminate\\Support\\Facades\\Redis","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to handle a newly introduced memory-logging lock mechanism.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Traits:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"WorkerIdTrait","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", along with the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"worker","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"field tracking in the final logs.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Loop and Worker Lifecycle Adjustments","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Loop and Worker Lifecycle Adjustments","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Worker Auto-Restart:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Deleted the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property, the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"constant, and the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"shouldIterateAgain()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method. The script no longer tracks execution iterations or triggers a self-managed worker restart.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Simplified Chunk Method Return:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Modified","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to always return","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Priority Handling for Throttled Entities","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Priority Handling for Throttled Entities","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Stale Record Elevation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Updated the logic in","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"rescheduleThrottledEntities()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to check if a throttled entity group is classified as \"stale\". If it is, the entities are rescheduled using","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"QueuePriorityEnum::NORMAL","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"; otherwise, they keep their respective queue priority.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Metrics, Logging, and Environment Handling","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Metrics, Logging, and Environment Handling","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Environment Tracking:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Introduced a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$environment","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property (defaulting to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"'local'","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") and a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"setEnvironment()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method to dynamically assign it.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Memory Monitoring via Datadog Gauge:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method that submits peak memory allocation to Datadog as a gauge metric (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"jiminny.reindex-memory-usage","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis Lock for Metrics Throttling:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Implemented a Redis-backed lock (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"%s-process-manager-lock","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") with a 60-second expiration inside","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to prevent over-reporting metrics.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Log Payload Update:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added total processed items (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$updatedDocuments + $deletedDocuments","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") to the final information log.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":23,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"You said are there any issues or potential bottlenecks","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"are there any issues or potential bottlenecks","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Edit","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a breakdown of the potential issues to look out for:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"1. Memory Leak Risk (OOM Crashes)","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"1. Memory Leak Risk (OOM Crashes)","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Restart Logic:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The most significant change is the removal of the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Dependency Leaks:","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.050033245,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If any underlying service (like the","depth":26,"bounds":{"left":0.17453457,"top":0.0,"width":0.08277926,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ElasticSearchClient","depth":27,"bounds":{"left":0.25930852,"top":0.0,"width":0.056848403,"height":0.015163607},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", ORM, or Datadog helper) retains references to processed entities,","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.21775267,"height":0.03631285},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"bounds":{"left":0.26811835,"top":0.0,"width":0.056848403,"height":0.015163607},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.23138298,"height":0.05546688},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Hardcoded Limits:","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.047041222,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The code still includes","depth":26,"bounds":{"left":0.17154256,"top":0.0,"width":0.05651596,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ini_set('memory_limit', '512M');","depth":27,"bounds":{"left":0.23005319,"top":0.0,"width":0.095744684,"height":0.015163607},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":". Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.23055187,"height":0.05546688},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"2. Race Condition in Redis Lock","depth":23,"bounds":{"left":0.111369684,"top":0.07940942,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"2. Race Condition in Redis Lock","depth":24,"bounds":{"left":0.111369684,"top":0.079010375,"width":0.09125665,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Non-Atomic Operations:","depth":26,"bounds":{"left":0.12450133,"top":0.10614525,"width":0.062333778,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The","depth":26,"bounds":{"left":0.18683511,"top":0.10614525,"width":0.011635638,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage","depth":27,"bounds":{"left":0.20046543,"top":0.10774142,"width":0.041888297,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method uses three distinct steps to handle the lock:","depth":26,"bounds":{"left":0.12450133,"top":0.10614525,"width":0.22722739,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"get()","depth":27,"bounds":{"left":0.14860372,"top":0.12689546,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":",","depth":26,"bounds":{"left":0.16555852,"top":0.12529927,"width":0.002493351,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"bounds":{"left":0.17004654,"top":0.12689546,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", and","depth":26,"bounds":{"left":0.18700133,"top":0.12529927,"width":0.013131649,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"expire()","depth":27,"bounds":{"left":0.20212767,"top":0.12689546,"width":0.023936171,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"bounds":{"left":0.22805852,"top":0.12529927,"width":0.0013297872,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"bounds":{"left":0.12450133,"top":0.15403032,"width":0.040059842,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If multiple workers process the same","depth":26,"bounds":{"left":0.16456117,"top":0.15403032,"width":0.09275266,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$this->updateTarget","depth":27,"bounds":{"left":0.25930852,"top":0.15562649,"width":0.056848403,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"concurrently, they can both pass the","depth":26,"bounds":{"left":0.12450133,"top":0.15403032,"width":0.22855718,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"if (! is_null(Redis::get(...)))","depth":27,"bounds":{"left":0.18351063,"top":0.17478053,"width":0.09275266,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"check at the exact same millisecond. Both will then execute","depth":26,"bounds":{"left":0.12450133,"top":0.17318435,"width":0.21575798,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"bounds":{"left":0.2122673,"top":0.19393456,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and push duplicate metrics to Datadog.","depth":26,"bounds":{"left":0.22922207,"top":0.19233839,"width":0.09823803,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Fix:","depth":26,"bounds":{"left":0.12450133,"top":0.22106944,"width":0.019281914,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"This should be replaced with a single atomic operation using Redis's","depth":26,"bounds":{"left":0.14378324,"top":0.22106944,"width":0.16821809,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"NX","depth":27,"bounds":{"left":0.31399602,"top":0.22266561,"width":0.005984043,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Not eXists) and","depth":26,"bounds":{"left":0.12450133,"top":0.22106944,"width":0.22855718,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"EX","depth":27,"bounds":{"left":0.13696809,"top":0.24181964,"width":0.005984043,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Expire) flags, or Laravel’s built-in atomic cache locks:","depth":26,"bounds":{"left":0.14494681,"top":0.24022347,"width":0.13264628,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis::set($gaugeLockName, true, 'EX', 60, 'NX')","depth":27,"bounds":{"left":0.12649602,"top":0.26097366,"width":0.14361702,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"bounds":{"left":0.27210772,"top":0.25937748,"width":0.0013297872,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"3. Infinite Loops and Caller Contract","depth":23,"bounds":{"left":0.111369684,"top":0.30925778,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"3. Infinite Loops and Caller Contract","depth":24,"bounds":{"left":0.111369684,"top":0.30885875,"width":0.10571808,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Return Value Change:","depth":26,"bounds":{"left":0.12450133,"top":0.33599362,"width":0.055352394,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"bounds":{"left":0.18301196,"top":0.33758977,"width":0.06881649,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"previously returned whether it should iterate again based on staleness or chunk size. It now always returns","depth":26,"bounds":{"left":0.12450133,"top":0.33599362,"width":0.2237367,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"bounds":{"left":0.29438165,"top":0.3567438,"width":0.011968086,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(unless the selection is totally empty).","depth":26,"bounds":{"left":0.12450133,"top":0.35514766,"width":0.2130984,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"bounds":{"left":0.12450133,"top":0.40303272,"width":0.040059842,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the code calling this method relies on a","depth":26,"bounds":{"left":0.16456117,"top":0.40303272,"width":0.10305851,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"false","depth":27,"bounds":{"left":0.26961437,"top":0.4046289,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"return to pause, sleep, or gracefully terminate, changing this to an unconditional","depth":26,"bounds":{"left":0.12450133,"top":0.40303272,"width":0.22589761,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"bounds":{"left":0.26163563,"top":0.4237829,"width":0.011968086,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"could result in an aggressive \"busy loop\" that spikes CPU usage.","depth":26,"bounds":{"left":0.12450133,"top":0.42218676,"width":0.22290559,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"4. Performance Overhead","depth":23,"bounds":{"left":0.111369684,"top":0.49122107,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"4. Performance Overhead","depth":24,"bounds":{"left":0.111369684,"top":0.49082202,"width":0.076130316,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Expensive Garbage Collection:","depth":26,"bounds":{"left":0.12450133,"top":0.5179569,"width":0.078457445,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Calling","depth":26,"bounds":{"left":0.20295878,"top":0.5179569,"width":0.019281914,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles();","depth":27,"bounds":{"left":0.22423537,"top":0.51955307,"width":0.059840426,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"on","depth":26,"bounds":{"left":0.28607047,"top":0.5179569,"width":0.00880984,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"every single chunk","depth":26,"bounds":{"left":0.29488033,"top":0.5179569,"width":0.045545213,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.","depth":26,"bounds":{"left":0.12450133,"top":0.5179569,"width":0.23420878,"height":0.05546688},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Synchronous Metric Dispatch:","depth":26,"bounds":{"left":0.12450133,"top":0.584996,"width":0.07712766,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the","depth":26,"bounds":{"left":0.20162898,"top":0.584996,"width":0.015292553,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::increment","depth":27,"bounds":{"left":0.21891622,"top":0.5865922,"width":0.053856384,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"or","depth":26,"bounds":{"left":0.27476728,"top":0.584996,"width":0.0078125,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::gauge","depth":27,"bounds":{"left":0.28457448,"top":0.5865922,"width":0.041888297,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.","depth":26,"bounds":{"left":0.12450133,"top":0.584996,"width":0.23420878,"height":0.05546688},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"5. Lack of Explicit Error Handling","depth":23,"bounds":{"left":0.111369684,"top":0.67318434,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5. Lack of Explicit Error Handling","depth":24,"bounds":{"left":0.111369684,"top":0.67278534,"width":0.094913565,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"No","depth":26,"bounds":{"left":0.12450133,"top":0.6999202,"width":0.008477394,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"try/catch","depth":27,"bounds":{"left":0.1349734,"top":0.7015164,"width":0.026928192,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Blocks:","depth":26,"bounds":{"left":0.16389628,"top":0.6999202,"width":0.019614361,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The script executes complex network operations (","depth":26,"bounds":{"left":0.18351063,"top":0.6999202,"width":0.12367021,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"loadDocuments","depth":27,"bounds":{"left":0.30917552,"top":0.7015164,"width":0.038896278,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.","depth":26,"bounds":{"left":0.12450133,"top":0.6999202,"width":0.22888963,"height":0.09377494},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":22,"bounds":{"left":0.1087101,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":22,"bounds":{"left":0.124667555,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Redo","depth":22,"bounds":{"left":0.140625,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":21,"bounds":{"left":0.15658244,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":22,"bounds":{"left":0.17253989,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":21,"bounds":{"left":0.18849733,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXTextArea","text":"Ask Gemini","depth":21,"bounds":{"left":0.14594415,"top":0.86632085,"width":0.17021276,"height":0.01915403},"on_screen":true,"value":"Ask Gemini","help_text":"","role_description":"text entry area","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Ask Gemini","depth":22,"bounds":{"left":0.14594415,"top":0.8667199,"width":0.030418882,"height":0.018355945},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Upload & tools","depth":20,"bounds":{"left":0.12732713,"top":0.8567438,"width":0.015957447,"height":0.03830806},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Microphone","depth":20,"bounds":{"left":0.32413563,"top":0.86153233,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Send message","depth":20,"bounds":{"left":0.33211437,"top":0.8631285,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":false,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.","depth":17,"bounds":{"left":0.102726065,"top":0.9173983,"width":0.21775267,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Your privacy & Gemini Opens in a new window","depth":17,"bounds":{"left":0.32047874,"top":0.9173983,"width":0.043882977,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Your privacy & Gemini","depth":18,"bounds":{"left":0.32047874,"top":0.9173983,"width":0.043882977,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Opens in a new window","depth":19,"bounds":{"left":0.079288565,"top":0.91660017,"width":0.04720745,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Summarize page","depth":7,"bounds":{"left":0.08494016,"top":0.95730245,"width":0.053523935,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Summarize page","depth":9,"bounds":{"left":0.09059176,"top":0.96249,"width":0.042220745,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Skip to content","depth":6,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Skip to content","depth":7,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Open menu","depth":10,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"Homepage (g then d)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"jiminny","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"jiminny","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"app","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"app","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Search or jump to…","depth":9,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Type","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"/","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to search","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Chat with Copilot","depth":10,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXMenuButton","text":"Open Copilot…","depth":9,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXMenuButton","text":"Create new...","depth":9,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"All issues(g then i)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"All pull requests","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"All repositories","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"You have unread notifications(g then n)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Open user navigation menu","depth":9,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Repository navigation","depth":9,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Repository navigation","depth":10,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Code","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Code","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Pull requests (32)","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Pull requests","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"32","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Agents","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Agents","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Actions","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Actions","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Wiki","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Wiki","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Security and quality","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Security and quality","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Insights","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Insights","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Settings","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Settings","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"JY-20915 add alias for EU #12105 Edit title","depth":13,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"JY-20915 add alias for EU","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"#","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"12105","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Edit title","depth":14,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Awaiting approval","depth":13,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Awaiting approval","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXMenuButton","text":"Code","depth":13,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXStaticText","text":"Code","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Open","depth":13,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"LakyLak","depth":15,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"wants to merge 5 commits into","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"master","depth":15,"on_screen":false,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"master","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"from","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915-add-domain-specific-email-text-relay","depth":16,"on_screen":false,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915-add-domain-specific-email-text-relay","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy head branch name to clipboard","depth":16,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Lines changed: 365 additions & 1 deletion","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Conversation (1)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"Conversation","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"1","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Commits (5)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Commits","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Checks (3)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Checks","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"3","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Files changed (4)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Files changed","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"4","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Open","depth":14,"bounds":{"left":0.50332445,"top":0.07222666,"width":0.011968086,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"JY-20915 add alias for EU #12105 LakyLak wants to merge 5 commits into master from JY-20915-add-domain-specific-email-text-relay Copy head branch name to clipboard","depth":14,"bounds":{"left":0.5219415,"top":0.058260176,"width":0.22922207,"height":0.042298485},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915 add alias for EU","depth":16,"bounds":{"left":0.5219415,"top":0.05865922,"width":0.05817819,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915 add alias for EU","depth":17,"bounds":{"left":0.5219415,"top":0.06304868,"width":0.05817819,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"}]...
|
567382309264853673
|
-1052796009668380900
|
idle
|
accessibility
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo
Share & export
Copy
Show more options
Ask Gemini
Ask Gemini
Upload & tools
Microphone
Send message
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Summarize page
Summarize page
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
All issues(g then i)
All pull requests
All repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Pull requests (32)
Pull requests
(
32
)
Agents
Agents
Actions
Actions
Wiki
Wiki
Security and quality
Security and quality
Insights
Insights
Settings
Settings
JY-20915 add alias for EU #12105 Edit title
JY-20915 add alias for EU
#
12105
Edit title
Awaiting approval
Awaiting approval
Code
Code
Open
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Lines changed: 365 additions & 1 deletion
Conversation (1)
Conversation
(
1
)
Commits (5)
Commits
(
5
)
Checks (3)
Checks
(
3
)
Files changed (4)
Files changed
(
4
)
Open
JY-20915 add alias for EU #12105 LakyLak wants to merge 5 commits into master from JY-20915-add-domain-specific-email-text-relay Copy head branch name to clipboard
JY-20915 add alias for EU
JY-20915 add alias for EU...
|
65258
|
NULL
|
NULL
|
NULL
|
|
65261
|
2289
|
4
|
2026-05-20T15:37:21.917698+00:00
|
/Users/lukas/.screenpipe/data/data/2026-05-20/1779 /Users/lukas/.screenpipe/data/data/2026-05-20/1779291441917_m1.jpg...
|
Firefox
|
JY-20915 add alias for EU by LakyLak · Pull Reques JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app — Work...
|
1
|
github.com/jiminny/app/pull/12105
|
monitor_1
|
NULL
|
NULL
|
NULL
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo
Share & export
Copy
Show more options
Ask Gemini
Ask Gemini
Upload & tools
Microphone
Send message
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Summarize page
Summarize page
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
All issues(g then i)
All pull requests
All repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Pull requests (32)
Pull requests
(
32
)
Agents
Agents
Actions
Actions
Wiki
Wiki
Security and quality
Security and quality
Insights
Insights
Settings
Settings
JY-20915 add alias for EU #12105 Edit title
JY-20915 add alias for EU
#
12105
Edit title
Awaiting approval
Awaiting approval
Code
Code
Open
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Lines changed: 365 additions & 1 deletion
Conversation (1)
Conversation
(
1
)
Commits (5)
Commits
(
5
)
Checks (3)
Checks...
|
[{"role":"AXRadioButton","text [{"role":"AXRadioButton","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Close tab","depth":5,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXRadioButton","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Pipelines - jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Pipelines - jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Tab","depth":4,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Customize sidebar","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Close Google Gemini (⌃X)","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Tabs from other devices","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open history (⇧⌘H)","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open bookmarks (⌘B)","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"AI Chat settings","depth":7,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Close","depth":7,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Main menu","depth":13,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"New chat","depth":12,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Gemini","depth":15,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Chat","depth":13,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Open menu for conversation actions.","depth":13,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Conversation with Gemini","depth":15,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Conversation with Gemini","depth":16,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"You said I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.","depth":23,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Expand","depth":21,"on_screen":false,"role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a concise summary of the changes made in the code selection:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Refactoring and Dependencies","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Refactoring and Dependencies","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Replaced Interface with Implementation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Changed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusInterface","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusManager","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"in the imports and constructor dependency injection.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Redis Support:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Imported","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Illuminate\\Support\\Facades\\Redis","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to handle a newly introduced memory-logging lock mechanism.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Traits:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"WorkerIdTrait","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", along with the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"worker","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"field tracking in the final logs.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Loop and Worker Lifecycle Adjustments","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Loop and Worker Lifecycle Adjustments","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Worker Auto-Restart:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Deleted the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property, the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"constant, and the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"shouldIterateAgain()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method. The script no longer tracks execution iterations or triggers a self-managed worker restart.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Simplified Chunk Method Return:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Modified","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to always return","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Priority Handling for Throttled Entities","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Priority Handling for Throttled Entities","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Stale Record Elevation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Updated the logic in","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"rescheduleThrottledEntities()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to check if a throttled entity group is classified as \"stale\". If it is, the entities are rescheduled using","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"QueuePriorityEnum::NORMAL","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"; otherwise, they keep their respective queue priority.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Metrics, Logging, and Environment Handling","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Metrics, Logging, and Environment Handling","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Environment Tracking:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Introduced a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$environment","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property (defaulting to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"'local'","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") and a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"setEnvironment()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method to dynamically assign it.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Memory Monitoring via Datadog Gauge:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method that submits peak memory allocation to Datadog as a gauge metric (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"jiminny.reindex-memory-usage","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis Lock for Metrics Throttling:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Implemented a Redis-backed lock (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"%s-process-manager-lock","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") with a 60-second expiration inside","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to prevent over-reporting metrics.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Log Payload Update:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added total processed items (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$updatedDocuments + $deletedDocuments","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") to the final information log.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":23,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"You said are there any issues or potential bottlenecks","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"are there any issues or potential bottlenecks","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Edit","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a breakdown of the potential issues to look out for:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"1. Memory Leak Risk (OOM Crashes)","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"1. Memory Leak Risk (OOM Crashes)","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Restart Logic:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The most significant change is the removal of the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Dependency Leaks:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If any underlying service (like the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ElasticSearchClient","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", ORM, or Datadog helper) retains references to processed entities,","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Hardcoded Limits:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The code still includes","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ini_set('memory_limit', '512M');","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":". Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"2. Race Condition in Redis Lock","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"2. Race Condition in Redis Lock","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Non-Atomic Operations:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method uses three distinct steps to handle the lock:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"get()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":",","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", and","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"expire()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If multiple workers process the same","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$this->updateTarget","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"concurrently, they can both pass the","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"if (! is_null(Redis::get(...)))","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"check at the exact same millisecond. Both will then execute","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and push duplicate metrics to Datadog.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Fix:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"This should be replaced with a single atomic operation using Redis's","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"NX","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Not eXists) and","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"EX","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Expire) flags, or Laravel’s built-in atomic cache locks:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis::set($gaugeLockName, true, 'EX', 60, 'NX')","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"3. Infinite Loops and Caller Contract","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"3. Infinite Loops and Caller Contract","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Return Value Change:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"previously returned whether it should iterate again based on staleness or chunk size. It now always returns","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(unless the selection is totally empty).","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the code calling this method relies on a","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"false","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"return to pause, sleep, or gracefully terminate, changing this to an unconditional","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"could result in an aggressive \"busy loop\" that spikes CPU usage.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"4. Performance Overhead","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"4. Performance Overhead","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Expensive Garbage Collection:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Calling","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles();","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"on","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"every single chunk","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Synchronous Metric Dispatch:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::increment","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"or","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::gauge","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"5. Lack of Explicit Error Handling","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5. Lack of Explicit Error Handling","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"No","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"try/catch","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Blocks:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The script executes complex network operations (","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"loadDocuments","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":22,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":22,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Redo","depth":22,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":21,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":22,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":21,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXTextArea","text":"Ask Gemini","depth":21,"on_screen":true,"value":"Ask Gemini","help_text":"","role_description":"text entry area","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Ask Gemini","depth":22,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Upload & tools","depth":20,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Microphone","depth":20,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Send message","depth":20,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":false,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.","depth":17,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Your privacy & Gemini Opens in a new window","depth":17,"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Your privacy & Gemini","depth":18,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Opens in a new window","depth":19,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Summarize page","depth":7,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Summarize page","depth":9,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Skip to content","depth":6,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Skip to content","depth":7,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Open menu","depth":10,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"Homepage (g then d)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"jiminny","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"jiminny","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"app","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"app","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Search or jump to…","depth":9,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Type","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"/","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to search","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Chat with Copilot","depth":10,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXMenuButton","text":"Open Copilot…","depth":9,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXMenuButton","text":"Create new...","depth":9,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"All issues(g then i)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"All pull requests","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"All repositories","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"You have unread notifications(g then n)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Open user navigation menu","depth":9,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Repository navigation","depth":9,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Repository navigation","depth":10,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Code","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Code","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Pull requests (32)","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Pull requests","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"32","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Agents","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Agents","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Actions","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Actions","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Wiki","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Wiki","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Security and quality","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Security and quality","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Insights","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Insights","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Settings","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Settings","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"JY-20915 add alias for EU #12105 Edit title","depth":13,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"JY-20915 add alias for EU","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"#","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"12105","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Edit title","depth":14,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Awaiting approval","depth":13,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Awaiting approval","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXMenuButton","text":"Code","depth":13,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXStaticText","text":"Code","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Open","depth":13,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"LakyLak","depth":15,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"wants to merge 5 commits into","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"master","depth":15,"on_screen":false,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"master","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"from","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915-add-domain-specific-email-text-relay","depth":16,"on_screen":false,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915-add-domain-specific-email-text-relay","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy head branch name to clipboard","depth":16,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Lines changed: 365 additions & 1 deletion","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Conversation (1)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"Conversation","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"1","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Commits (5)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Commits","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Checks (3)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Checks","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"}]...
|
-1972500944610712270
|
-1052936747159898340
|
idle
|
accessibility
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo
Share & export
Copy
Show more options
Ask Gemini
Ask Gemini
Upload & tools
Microphone
Send message
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Summarize page
Summarize page
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
All issues(g then i)
All pull requests
All repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Pull requests (32)
Pull requests
(
32
)
Agents
Agents
Actions
Actions
Wiki
Wiki
Security and quality
Security and quality
Insights
Insights
Settings
Settings
JY-20915 add alias for EU #12105 Edit title
JY-20915 add alias for EU
#
12105
Edit title
Awaiting approval
Awaiting approval
Code
Code
Open
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Lines changed: 365 additions & 1 deletion
Conversation (1)
Conversation
(
1
)
Commits (5)
Commits
(
5
)
Checks (3)
Checks...
|
65259
|
NULL
|
NULL
|
NULL
|
|
65262
|
2290
|
5
|
2026-05-20T15:37:34.748296+00:00
|
/Users/lukas/.screenpipe/data/data/2026-05-20/1779 /Users/lukas/.screenpipe/data/data/2026-05-20/1779291454748_m2.jpg...
|
Firefox
|
JY-20915 add alias for EU by LakyLak · Pull Reques JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app — Work...
|
1
|
github.com/jiminny/app/pull/12105
|
monitor_2
|
NULL
|
NULL
|
NULL
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo
Share & export
Copy
Show more options
Ask Gemini
Ask Gemini
Upload & tools
Microphone
Send message
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Summarize page
Summarize page
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
All issues(g then i)
All pull requests
All repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Pull requests (32)
Pull requests
(
32
)
Agents
Agents
Actions
Actions
Wiki
Wiki
Security and quality
Security and quality
Insights
Insights
Settings
Settings
JY-20915 add alias for EU #12105 Edit title
JY-20915 add alias for EU
#
12105...
|
[{"role":"AXRadioButton","text [{"role":"AXRadioButton","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":4,"bounds":{"left":0.0,"top":0.0518755,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.06304868,"width":0.10106383,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":4,"bounds":{"left":0.0,"top":0.08459697,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.09577015,"width":0.10721409,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.11731844,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.12849163,"width":0.15791224,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.15003991,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.16121309,"width":0.1796875,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.18276137,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.19393456,"width":0.12699468,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Close tab","depth":5,"bounds":{"left":0.06732048,"top":0.18994413,"width":0.007978723,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXRadioButton","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":4,"bounds":{"left":0.0,"top":0.21548285,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":5,"bounds":{"left":0.013297873,"top":0.22665602,"width":0.10139628,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":4,"bounds":{"left":0.0,"top":0.2482043,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.25937748,"width":0.18068483,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":4,"bounds":{"left":0.0,"top":0.28092578,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":5,"bounds":{"left":0.013297873,"top":0.29209897,"width":0.4644282,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Pipelines - jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.31364724,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Pipelines - jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.32482043,"width":0.039228722,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.3463687,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.3575419,"width":0.15824468,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Tab","depth":4,"bounds":{"left":0.0028257978,"top":0.38068634,"width":0.07413564,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Customize sidebar","depth":6,"bounds":{"left":0.0028257978,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Close Google Gemini (⌃X)","depth":6,"bounds":{"left":0.013796543,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Tabs from other devices","depth":6,"bounds":{"left":0.024933511,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open history (⇧⌘H)","depth":6,"bounds":{"left":0.036070477,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open bookmarks (⌘B)","depth":6,"bounds":{"left":0.04720745,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"AI Chat settings","depth":7,"bounds":{"left":0.35854387,"top":0.055067837,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Close","depth":7,"bounds":{"left":0.37051198,"top":0.055067837,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Main menu","depth":13,"bounds":{"left":0.08228058,"top":0.09736632,"width":0.015957447,"height":0.03830806},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"New chat","depth":12,"bounds":{"left":0.09823803,"top":0.10215483,"width":0.028091755,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Gemini","depth":15,"bounds":{"left":0.1008976,"top":0.10654429,"width":0.02144282,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Chat","depth":13,"bounds":{"left":0.35854387,"top":0.10215483,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Open menu for conversation actions.","depth":13,"bounds":{"left":0.37051198,"top":0.10215483,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Conversation with Gemini","depth":15,"bounds":{"left":0.079288565,"top":0.14445332,"width":0.0003324468,"height":0.0007980846},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Conversation with Gemini","depth":16,"bounds":{"left":0.079288565,"top":0.14684756,"width":0.1200133,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"You said I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.","depth":23,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Expand","depth":21,"on_screen":false,"role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a concise summary of the changes made in the code selection:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Refactoring and Dependencies","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Refactoring and Dependencies","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Replaced Interface with Implementation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Changed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusInterface","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusManager","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"in the imports and constructor dependency injection.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Redis Support:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Imported","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Illuminate\\Support\\Facades\\Redis","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to handle a newly introduced memory-logging lock mechanism.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Traits:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"WorkerIdTrait","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", along with the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"worker","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"field tracking in the final logs.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Loop and Worker Lifecycle Adjustments","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Loop and Worker Lifecycle Adjustments","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Worker Auto-Restart:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Deleted the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property, the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"constant, and the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"shouldIterateAgain()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method. The script no longer tracks execution iterations or triggers a self-managed worker restart.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Simplified Chunk Method Return:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Modified","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to always return","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Priority Handling for Throttled Entities","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Priority Handling for Throttled Entities","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Stale Record Elevation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Updated the logic in","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"rescheduleThrottledEntities()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to check if a throttled entity group is classified as \"stale\". If it is, the entities are rescheduled using","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"QueuePriorityEnum::NORMAL","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"; otherwise, they keep their respective queue priority.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Metrics, Logging, and Environment Handling","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Metrics, Logging, and Environment Handling","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Environment Tracking:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Introduced a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$environment","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property (defaulting to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"'local'","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") and a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"setEnvironment()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method to dynamically assign it.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Memory Monitoring via Datadog Gauge:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method that submits peak memory allocation to Datadog as a gauge metric (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"jiminny.reindex-memory-usage","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis Lock for Metrics Throttling:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Implemented a Redis-backed lock (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"%s-process-manager-lock","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") with a 60-second expiration inside","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to prevent over-reporting metrics.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Log Payload Update:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added total processed items (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$updatedDocuments + $deletedDocuments","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") to the final information log.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":23,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"You said are there any issues or potential bottlenecks","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"are there any issues or potential bottlenecks","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Edit","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a breakdown of the potential issues to look out for:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"1. Memory Leak Risk (OOM Crashes)","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"1. Memory Leak Risk (OOM Crashes)","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Restart Logic:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The most significant change is the removal of the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Dependency Leaks:","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.050033245,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If any underlying service (like the","depth":26,"bounds":{"left":0.17453457,"top":0.0,"width":0.08277926,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ElasticSearchClient","depth":27,"bounds":{"left":0.25930852,"top":0.0,"width":0.056848403,"height":0.015163607},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", ORM, or Datadog helper) retains references to processed entities,","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.21775267,"height":0.03631285},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"bounds":{"left":0.26811835,"top":0.0,"width":0.056848403,"height":0.015163607},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.23138298,"height":0.05546688},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Hardcoded Limits:","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.047041222,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The code still includes","depth":26,"bounds":{"left":0.17154256,"top":0.0,"width":0.05651596,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ini_set('memory_limit', '512M');","depth":27,"bounds":{"left":0.23005319,"top":0.0,"width":0.095744684,"height":0.015163607},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":". Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.23055187,"height":0.05546688},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"2. Race Condition in Redis Lock","depth":23,"bounds":{"left":0.111369684,"top":0.07940942,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"2. Race Condition in Redis Lock","depth":24,"bounds":{"left":0.111369684,"top":0.079010375,"width":0.09125665,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Non-Atomic Operations:","depth":26,"bounds":{"left":0.12450133,"top":0.10614525,"width":0.062333778,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The","depth":26,"bounds":{"left":0.18683511,"top":0.10614525,"width":0.011635638,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage","depth":27,"bounds":{"left":0.20046543,"top":0.10774142,"width":0.041888297,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method uses three distinct steps to handle the lock:","depth":26,"bounds":{"left":0.12450133,"top":0.10614525,"width":0.22722739,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"get()","depth":27,"bounds":{"left":0.14860372,"top":0.12689546,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":",","depth":26,"bounds":{"left":0.16555852,"top":0.12529927,"width":0.002493351,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"bounds":{"left":0.17004654,"top":0.12689546,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", and","depth":26,"bounds":{"left":0.18700133,"top":0.12529927,"width":0.013131649,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"expire()","depth":27,"bounds":{"left":0.20212767,"top":0.12689546,"width":0.023936171,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"bounds":{"left":0.22805852,"top":0.12529927,"width":0.0013297872,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"bounds":{"left":0.12450133,"top":0.15403032,"width":0.040059842,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If multiple workers process the same","depth":26,"bounds":{"left":0.16456117,"top":0.15403032,"width":0.09275266,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$this->updateTarget","depth":27,"bounds":{"left":0.25930852,"top":0.15562649,"width":0.056848403,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"concurrently, they can both pass the","depth":26,"bounds":{"left":0.12450133,"top":0.15403032,"width":0.22855718,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"if (! is_null(Redis::get(...)))","depth":27,"bounds":{"left":0.18351063,"top":0.17478053,"width":0.09275266,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"check at the exact same millisecond. Both will then execute","depth":26,"bounds":{"left":0.12450133,"top":0.17318435,"width":0.21575798,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"bounds":{"left":0.2122673,"top":0.19393456,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and push duplicate metrics to Datadog.","depth":26,"bounds":{"left":0.22922207,"top":0.19233839,"width":0.09823803,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Fix:","depth":26,"bounds":{"left":0.12450133,"top":0.22106944,"width":0.019281914,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"This should be replaced with a single atomic operation using Redis's","depth":26,"bounds":{"left":0.14378324,"top":0.22106944,"width":0.16821809,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"NX","depth":27,"bounds":{"left":0.31399602,"top":0.22266561,"width":0.005984043,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Not eXists) and","depth":26,"bounds":{"left":0.12450133,"top":0.22106944,"width":0.22855718,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"EX","depth":27,"bounds":{"left":0.13696809,"top":0.24181964,"width":0.005984043,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Expire) flags, or Laravel’s built-in atomic cache locks:","depth":26,"bounds":{"left":0.14494681,"top":0.24022347,"width":0.13264628,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis::set($gaugeLockName, true, 'EX', 60, 'NX')","depth":27,"bounds":{"left":0.12649602,"top":0.26097366,"width":0.14361702,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"bounds":{"left":0.27210772,"top":0.25937748,"width":0.0013297872,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"3. Infinite Loops and Caller Contract","depth":23,"bounds":{"left":0.111369684,"top":0.30925778,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"3. Infinite Loops and Caller Contract","depth":24,"bounds":{"left":0.111369684,"top":0.30885875,"width":0.10571808,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Return Value Change:","depth":26,"bounds":{"left":0.12450133,"top":0.33599362,"width":0.055352394,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"bounds":{"left":0.18301196,"top":0.33758977,"width":0.06881649,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"previously returned whether it should iterate again based on staleness or chunk size. It now always returns","depth":26,"bounds":{"left":0.12450133,"top":0.33599362,"width":0.2237367,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"bounds":{"left":0.29438165,"top":0.3567438,"width":0.011968086,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(unless the selection is totally empty).","depth":26,"bounds":{"left":0.12450133,"top":0.35514766,"width":0.2130984,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"bounds":{"left":0.12450133,"top":0.40303272,"width":0.040059842,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the code calling this method relies on a","depth":26,"bounds":{"left":0.16456117,"top":0.40303272,"width":0.10305851,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"false","depth":27,"bounds":{"left":0.26961437,"top":0.4046289,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"return to pause, sleep, or gracefully terminate, changing this to an unconditional","depth":26,"bounds":{"left":0.12450133,"top":0.40303272,"width":0.22589761,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"bounds":{"left":0.26163563,"top":0.4237829,"width":0.011968086,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"could result in an aggressive \"busy loop\" that spikes CPU usage.","depth":26,"bounds":{"left":0.12450133,"top":0.42218676,"width":0.22290559,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"4. Performance Overhead","depth":23,"bounds":{"left":0.111369684,"top":0.49122107,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"4. Performance Overhead","depth":24,"bounds":{"left":0.111369684,"top":0.49082202,"width":0.076130316,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Expensive Garbage Collection:","depth":26,"bounds":{"left":0.12450133,"top":0.5179569,"width":0.078457445,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Calling","depth":26,"bounds":{"left":0.20295878,"top":0.5179569,"width":0.019281914,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles();","depth":27,"bounds":{"left":0.22423537,"top":0.51955307,"width":0.059840426,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"on","depth":26,"bounds":{"left":0.28607047,"top":0.5179569,"width":0.00880984,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"every single chunk","depth":26,"bounds":{"left":0.29488033,"top":0.5179569,"width":0.045545213,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.","depth":26,"bounds":{"left":0.12450133,"top":0.5179569,"width":0.23420878,"height":0.05546688},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Synchronous Metric Dispatch:","depth":26,"bounds":{"left":0.12450133,"top":0.584996,"width":0.07712766,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the","depth":26,"bounds":{"left":0.20162898,"top":0.584996,"width":0.015292553,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::increment","depth":27,"bounds":{"left":0.21891622,"top":0.5865922,"width":0.053856384,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"or","depth":26,"bounds":{"left":0.27476728,"top":0.584996,"width":0.0078125,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::gauge","depth":27,"bounds":{"left":0.28457448,"top":0.5865922,"width":0.041888297,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.","depth":26,"bounds":{"left":0.12450133,"top":0.584996,"width":0.23420878,"height":0.05546688},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"5. Lack of Explicit Error Handling","depth":23,"bounds":{"left":0.111369684,"top":0.67318434,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5. Lack of Explicit Error Handling","depth":24,"bounds":{"left":0.111369684,"top":0.67278534,"width":0.094913565,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"No","depth":26,"bounds":{"left":0.12450133,"top":0.6999202,"width":0.008477394,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"try/catch","depth":27,"bounds":{"left":0.1349734,"top":0.7015164,"width":0.026928192,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Blocks:","depth":26,"bounds":{"left":0.16389628,"top":0.6999202,"width":0.019614361,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The script executes complex network operations (","depth":26,"bounds":{"left":0.18351063,"top":0.6999202,"width":0.12367021,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"loadDocuments","depth":27,"bounds":{"left":0.30917552,"top":0.7015164,"width":0.038896278,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.","depth":26,"bounds":{"left":0.12450133,"top":0.6999202,"width":0.22888963,"height":0.09377494},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":22,"bounds":{"left":0.1087101,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":22,"bounds":{"left":0.124667555,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Redo","depth":22,"bounds":{"left":0.140625,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":21,"bounds":{"left":0.15658244,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":22,"bounds":{"left":0.17253989,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":21,"bounds":{"left":0.18849733,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXTextArea","text":"Ask Gemini","depth":21,"bounds":{"left":0.14594415,"top":0.86632085,"width":0.17021276,"height":0.01915403},"on_screen":true,"value":"Ask Gemini","help_text":"","role_description":"text entry area","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Ask Gemini","depth":22,"bounds":{"left":0.14594415,"top":0.8667199,"width":0.030418882,"height":0.018355945},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Upload & tools","depth":20,"bounds":{"left":0.12732713,"top":0.8567438,"width":0.015957447,"height":0.03830806},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Microphone","depth":20,"bounds":{"left":0.32413563,"top":0.86153233,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Send message","depth":20,"bounds":{"left":0.33211437,"top":0.8631285,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":false,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.","depth":17,"bounds":{"left":0.102726065,"top":0.9173983,"width":0.21775267,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Your privacy & Gemini Opens in a new window","depth":17,"bounds":{"left":0.32047874,"top":0.9173983,"width":0.043882977,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Your privacy & Gemini","depth":18,"bounds":{"left":0.32047874,"top":0.9173983,"width":0.043882977,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Opens in a new window","depth":19,"bounds":{"left":0.079288565,"top":0.91660017,"width":0.04720745,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Summarize page","depth":7,"bounds":{"left":0.08494016,"top":0.95730245,"width":0.053523935,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Summarize page","depth":9,"bounds":{"left":0.09059176,"top":0.96249,"width":0.042220745,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Skip to content","depth":6,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Skip to content","depth":7,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Open menu","depth":10,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"Homepage (g then d)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"jiminny","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"jiminny","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"app","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"app","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Search or jump to…","depth":9,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Type","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"/","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to search","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Chat with Copilot","depth":10,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXMenuButton","text":"Open Copilot…","depth":9,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXMenuButton","text":"Create new...","depth":9,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"All issues(g then i)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"All pull requests","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"All repositories","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"You have unread notifications(g then n)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Open user navigation menu","depth":9,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Repository navigation","depth":9,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Repository navigation","depth":10,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Code","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Code","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Pull requests (32)","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Pull requests","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"32","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Agents","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Agents","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Actions","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Actions","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Wiki","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Wiki","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Security and quality","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Security and quality","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Insights","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Insights","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Settings","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Settings","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"JY-20915 add alias for EU #12105 Edit title","depth":13,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"JY-20915 add alias for EU","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"#","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"12105","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"}]...
|
2993435265994294713
|
-1052866378415638756
|
idle
|
accessibility
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo
Share & export
Copy
Show more options
Ask Gemini
Ask Gemini
Upload & tools
Microphone
Send message
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Summarize page
Summarize page
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
All issues(g then i)
All pull requests
All repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Pull requests (32)
Pull requests
(
32
)
Agents
Agents
Actions
Actions
Wiki
Wiki
Security and quality
Security and quality
Insights
Insights
Settings
Settings
JY-20915 add alias for EU #12105 Edit title
JY-20915 add alias for EU
#
12105...
|
NULL
|
NULL
|
NULL
|
NULL
|
|
65263
|
2289
|
5
|
2026-05-20T15:37:52.275815+00:00
|
/Users/lukas/.screenpipe/data/data/2026-05-20/1779 /Users/lukas/.screenpipe/data/data/2026-05-20/1779291472275_m1.jpg...
|
Firefox
|
JY-20915 add alias for EU by LakyLak · Pull Reques JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app — Work...
|
1
|
github.com/jiminny/app/pull/12105
|
monitor_1
|
NULL
|
NULL
|
NULL
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo
Share & export
Copy
Show more options
Ask Gemini
Ask Gemini
Upload & tools
Microphone
Send message
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Summarize page
Summarize page
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
All issues(g then i)
All pull requests
All repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Pull requests (32)
Pull requests
(
32
)
Agents
Agents
Actions
Actions
Wiki
Wiki
Security and quality
Security and quality
Insights
Insights
Settings
Settings
JY-20915 add alias for EU #12105 Edit title
JY-20915 add alias for EU
#
12105
Edit title
Awaiting approval
Awaiting approval
Code
Code
Open
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Lines changed: 365 additions & 1 deletion
Conversation (1)
Conversation
(
1
)
Commits (5)
Commits
(
5
)
Checks (3)
Checks
(
3
)
Files changed (4)
Files changed
(
4
)
Open
JY-20915 add alias for EU #12105 LakyLak wants to merge 5 commits into master from JY-20915-add-domain-specific-email-text-relay Copy head branch name to clipboard
JY-20915 add alias for EU
JY-20915 add alias for EU
#
12105
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Conversation
Conversation
@LakyLak
Show options
LakyLak commented yesterday
LakyLak
LakyLak
commented
yesterday
yesterday
JIRA: JY-20915
JIRA:
JY-20915
JY-20915
Changes:
Changes:
Support text relay for EU
Prevent errors from EU on US
Add or remove reactions
LakyLak
LakyLak
added
5
commits
yesterday
yesterday
@LakyLak
JY-20915
JY-20915
add alias for EU
add alias for EU
8 / 10 checks OK
e5255d5
e5255d5
@LakyLak
JY-20915
JY-20915
fix env recognization
fix env recognization
11 / 12 checks OK
8953054
8953054
@LakyLak
JY-20915
JY-20915
code smells & tests
code smells & tests
11 / 12 checks OK
73d4bea
73d4bea
@LakyLak
JY-20915
JY-20915
add more tests
add more tests
11 / 12 checks OK
5d2dd3e
5d2dd3e
@LakyLak...
|
[{"role":"AXRadioButton","text [{"role":"AXRadioButton","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Close tab","depth":5,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXRadioButton","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Pipelines - jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Pipelines - jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Tab","depth":4,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Customize sidebar","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Close Google Gemini (⌃X)","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Tabs from other devices","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open history (⇧⌘H)","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open bookmarks (⌘B)","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"AI Chat settings","depth":7,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Close","depth":7,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Main menu","depth":13,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"New chat","depth":12,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Gemini","depth":15,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Chat","depth":13,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Open menu for conversation actions.","depth":13,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Conversation with Gemini","depth":15,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Conversation with Gemini","depth":16,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"You said I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.","depth":23,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Expand","depth":21,"on_screen":false,"role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a concise summary of the changes made in the code selection:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Refactoring and Dependencies","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Refactoring and Dependencies","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Replaced Interface with Implementation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Changed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusInterface","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusManager","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"in the imports and constructor dependency injection.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Redis Support:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Imported","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Illuminate\\Support\\Facades\\Redis","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to handle a newly introduced memory-logging lock mechanism.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Traits:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"WorkerIdTrait","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", along with the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"worker","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"field tracking in the final logs.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Loop and Worker Lifecycle Adjustments","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Loop and Worker Lifecycle Adjustments","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Worker Auto-Restart:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Deleted the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property, the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"constant, and the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"shouldIterateAgain()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method. The script no longer tracks execution iterations or triggers a self-managed worker restart.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Simplified Chunk Method Return:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Modified","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to always return","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Priority Handling for Throttled Entities","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Priority Handling for Throttled Entities","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Stale Record Elevation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Updated the logic in","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"rescheduleThrottledEntities()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to check if a throttled entity group is classified as \"stale\". If it is, the entities are rescheduled using","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"QueuePriorityEnum::NORMAL","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"; otherwise, they keep their respective queue priority.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Metrics, Logging, and Environment Handling","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Metrics, Logging, and Environment Handling","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Environment Tracking:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Introduced a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$environment","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property (defaulting to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"'local'","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") and a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"setEnvironment()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method to dynamically assign it.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Memory Monitoring via Datadog Gauge:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method that submits peak memory allocation to Datadog as a gauge metric (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"jiminny.reindex-memory-usage","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis Lock for Metrics Throttling:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Implemented a Redis-backed lock (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"%s-process-manager-lock","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") with a 60-second expiration inside","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to prevent over-reporting metrics.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Log Payload Update:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added total processed items (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$updatedDocuments + $deletedDocuments","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") to the final information log.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":23,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"You said are there any issues or potential bottlenecks","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"are there any issues or potential bottlenecks","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Edit","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a breakdown of the potential issues to look out for:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"1. Memory Leak Risk (OOM Crashes)","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"1. Memory Leak Risk (OOM Crashes)","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Restart Logic:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The most significant change is the removal of the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Dependency Leaks:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If any underlying service (like the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ElasticSearchClient","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", ORM, or Datadog helper) retains references to processed entities,","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Hardcoded Limits:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The code still includes","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ini_set('memory_limit', '512M');","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":". Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"2. Race Condition in Redis Lock","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"2. Race Condition in Redis Lock","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Non-Atomic Operations:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method uses three distinct steps to handle the lock:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"get()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":",","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", and","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"expire()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If multiple workers process the same","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$this->updateTarget","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"concurrently, they can both pass the","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"if (! is_null(Redis::get(...)))","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"check at the exact same millisecond. Both will then execute","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and push duplicate metrics to Datadog.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Fix:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"This should be replaced with a single atomic operation using Redis's","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"NX","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Not eXists) and","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"EX","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Expire) flags, or Laravel’s built-in atomic cache locks:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis::set($gaugeLockName, true, 'EX', 60, 'NX')","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"3. Infinite Loops and Caller Contract","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"3. Infinite Loops and Caller Contract","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Return Value Change:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"previously returned whether it should iterate again based on staleness or chunk size. It now always returns","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(unless the selection is totally empty).","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the code calling this method relies on a","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"false","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"return to pause, sleep, or gracefully terminate, changing this to an unconditional","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"could result in an aggressive \"busy loop\" that spikes CPU usage.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"4. Performance Overhead","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"4. Performance Overhead","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Expensive Garbage Collection:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Calling","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles();","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"on","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"every single chunk","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Synchronous Metric Dispatch:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::increment","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"or","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::gauge","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"5. Lack of Explicit Error Handling","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5. Lack of Explicit Error Handling","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"No","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"try/catch","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Blocks:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The script executes complex network operations (","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"loadDocuments","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":22,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":22,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Redo","depth":22,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":21,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":22,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":21,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXTextArea","text":"Ask Gemini","depth":21,"on_screen":true,"value":"Ask Gemini","help_text":"","role_description":"text entry area","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Ask Gemini","depth":22,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Upload & tools","depth":20,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Microphone","depth":20,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Send message","depth":20,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":false,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.","depth":17,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Your privacy & Gemini Opens in a new window","depth":17,"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Your privacy & Gemini","depth":18,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Opens in a new window","depth":19,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Summarize page","depth":7,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Summarize page","depth":9,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Skip to content","depth":6,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Skip to content","depth":7,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Open menu","depth":10,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"Homepage (g then d)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"jiminny","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"jiminny","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"app","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"app","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Search or jump to…","depth":9,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Type","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"/","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to search","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Chat with Copilot","depth":10,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXMenuButton","text":"Open Copilot…","depth":9,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXMenuButton","text":"Create new...","depth":9,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"All issues(g then i)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"All pull requests","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"All repositories","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"You have unread notifications(g then n)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Open user navigation menu","depth":9,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Repository navigation","depth":9,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Repository navigation","depth":10,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Code","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Code","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Pull requests (32)","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Pull requests","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"32","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Agents","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Agents","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Actions","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Actions","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Wiki","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Wiki","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Security and quality","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Security and quality","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Insights","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Insights","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Settings","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Settings","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"JY-20915 add alias for EU #12105 Edit title","depth":13,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"JY-20915 add alias for EU","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"#","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"12105","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Edit title","depth":14,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Awaiting approval","depth":13,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Awaiting approval","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXMenuButton","text":"Code","depth":13,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXStaticText","text":"Code","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Open","depth":13,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"LakyLak","depth":15,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"wants to merge 5 commits into","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"master","depth":15,"on_screen":false,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"master","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"from","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915-add-domain-specific-email-text-relay","depth":16,"on_screen":false,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915-add-domain-specific-email-text-relay","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy head branch name to clipboard","depth":16,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Lines changed: 365 additions & 1 deletion","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Conversation (1)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"Conversation","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"1","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Commits (5)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Commits","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Checks (3)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Checks","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"3","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Files changed (4)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Files changed","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"4","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Open","depth":14,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"JY-20915 add alias for EU #12105 LakyLak wants to merge 5 commits into master from JY-20915-add-domain-specific-email-text-relay Copy head branch name to clipboard","depth":14,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915 add alias for EU","depth":16,"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915 add alias for EU","depth":17,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"#","depth":16,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"12105","depth":16,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"LakyLak","depth":18,"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":19,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"wants to merge 5 commits into","depth":18,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"master","depth":18,"on_screen":true,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"master","depth":19,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"from","depth":19,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915-add-domain-specific-email-text-relay","depth":19,"on_screen":true,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915-add-domain-specific-email-text-relay","depth":20,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy head branch name to clipboard","depth":19,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Conversation","depth":12,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Conversation","depth":13,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@LakyLak","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show options","depth":15,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXHeading","text":"LakyLak commented yesterday","depth":14,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXLink","text":"LakyLak","depth":16,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"commented","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"yesterday","depth":15,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"yesterday","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"JIRA: JY-20915","depth":16,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"JIRA:","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915","depth":17,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Changes:","depth":16,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Changes:","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Support text relay for EU","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Prevent errors from EU on US","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Add or remove reactions","depth":16,"on_screen":false,"help_text":"","role_description":"summary","subrole":"AXSummary","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"LakyLak","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"added","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"commits","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"yesterday","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"yesterday","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@LakyLak","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"JY-20915","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"add alias for EU","depth":14,"on_screen":false,"help_text":"JY-20915 add alias for EU","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"add alias for EU","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"8 / 10 checks OK","depth":14,"on_screen":false,"help_text":"","role_description":"summary","subrole":"AXSummary","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"e5255d5","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"e5255d5","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@LakyLak","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"JY-20915","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"fix env recognization","depth":14,"on_screen":false,"help_text":"JY-20915 fix env recognization","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"fix env recognization","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"11 / 12 checks OK","depth":14,"on_screen":false,"help_text":"","role_description":"summary","subrole":"AXSummary","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"8953054","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"8953054","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@LakyLak","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"JY-20915","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"code smells & tests","depth":14,"on_screen":false,"help_text":"JY-20915 code smells & tests","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"code smells & tests","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"11 / 12 checks OK","depth":14,"on_screen":false,"help_text":"","role_description":"summary","subrole":"AXSummary","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"73d4bea","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"73d4bea","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@LakyLak","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"JY-20915","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"add more tests","depth":14,"on_screen":false,"help_text":"JY-20915 add more tests","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"add more tests","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"11 / 12 checks OK","depth":14,"on_screen":false,"help_text":"","role_description":"summary","subrole":"AXSummary","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"5d2dd3e","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"5d2dd3e","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@LakyLak","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false}]...
|
1168576886443098558
|
-1048292410041026792
|
idle
|
accessibility
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo
Share & export
Copy
Show more options
Ask Gemini
Ask Gemini
Upload & tools
Microphone
Send message
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Summarize page
Summarize page
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
All issues(g then i)
All pull requests
All repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Pull requests (32)
Pull requests
(
32
)
Agents
Agents
Actions
Actions
Wiki
Wiki
Security and quality
Security and quality
Insights
Insights
Settings
Settings
JY-20915 add alias for EU #12105 Edit title
JY-20915 add alias for EU
#
12105
Edit title
Awaiting approval
Awaiting approval
Code
Code
Open
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Lines changed: 365 additions & 1 deletion
Conversation (1)
Conversation
(
1
)
Commits (5)
Commits
(
5
)
Checks (3)
Checks
(
3
)
Files changed (4)
Files changed
(
4
)
Open
JY-20915 add alias for EU #12105 LakyLak wants to merge 5 commits into master from JY-20915-add-domain-specific-email-text-relay Copy head branch name to clipboard
JY-20915 add alias for EU
JY-20915 add alias for EU
#
12105
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Conversation
Conversation
@LakyLak
Show options
LakyLak commented yesterday
LakyLak
LakyLak
commented
yesterday
yesterday
JIRA: JY-20915
JIRA:
JY-20915
JY-20915
Changes:
Changes:
Support text relay for EU
Prevent errors from EU on US
Add or remove reactions
LakyLak
LakyLak
added
5
commits
yesterday
yesterday
@LakyLak
JY-20915
JY-20915
add alias for EU
add alias for EU
8 / 10 checks OK
e5255d5
e5255d5
@LakyLak
JY-20915
JY-20915
fix env recognization
fix env recognization
11 / 12 checks OK
8953054
8953054
@LakyLak
JY-20915
JY-20915
code smells & tests
code smells & tests
11 / 12 checks OK
73d4bea
73d4bea
@LakyLak
JY-20915
JY-20915
add more tests
add more tests
11 / 12 checks OK
5d2dd3e
5d2dd3e
@LakyLak...
|
NULL
|
NULL
|
NULL
|
NULL
|
|
65264
|
2290
|
6
|
2026-05-20T15:38:05.179876+00:00
|
/Users/lukas/.screenpipe/data/data/2026-05-20/1779 /Users/lukas/.screenpipe/data/data/2026-05-20/1779291485179_m2.jpg...
|
Firefox
|
JY-20915 add alias for EU by LakyLak · Pull Reques JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app — Work...
|
1
|
github.com/jiminny/app/pull/12105
|
monitor_2
|
NULL
|
NULL
|
NULL
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo
Share & export
Copy
Show more options
Ask Gemini
Ask Gemini
Upload & tools
Microphone
Send message
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Summarize page
Summarize page
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
All issues(g then i)
All pull requests
All repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Pull requests (32)
Pull requests
(
32
)
Agents
Agents
Actions
Actions
Wiki
Wiki
Security and quality
Security and quality
Insights
Insights
Settings
Settings
JY-20915 add alias for EU #12105 Edit title
JY-20915 add alias for EU
#
12105
Edit title
Awaiting approval
Awaiting approval
Code
Code
Open
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Lines changed: 365 additions & 1 deletion
Conversation (1)
Conversation
(
1
)
Commits (5)
Commits
(
5
)
Checks (3)
Checks
(
3
)
Files changed (4)
Files changed
(
4
)
Open
JY-20915 add alias for EU #12105 LakyLak wants to merge 5 commits into master from JY-20915-add-domain-specific-email-text-relay Copy head branch name to clipboard
JY-20915 add alias for EU
JY-20915 add alias for EU
#
12105
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Conversation
Conversation
@LakyLak
Show options
LakyLak commented yesterday
LakyLak
LakyLak
commented
yesterday
yesterday
JIRA: JY-20915
JIRA:
JY-20915
JY-20915
Changes:
Changes:
Support text relay for EU
Prevent errors from EU on US...
|
[{"role":"AXRadioButton","text [{"role":"AXRadioButton","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":4,"bounds":{"left":0.0,"top":0.0518755,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.06304868,"width":0.10106383,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":4,"bounds":{"left":0.0,"top":0.08459697,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.09577015,"width":0.10721409,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.11731844,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.12849163,"width":0.15791224,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.15003991,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.16121309,"width":0.1796875,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.18276137,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.19393456,"width":0.12699468,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Close tab","depth":5,"bounds":{"left":0.06732048,"top":0.18994413,"width":0.007978723,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXRadioButton","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":4,"bounds":{"left":0.0,"top":0.21548285,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":5,"bounds":{"left":0.013297873,"top":0.22665602,"width":0.10139628,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":4,"bounds":{"left":0.0,"top":0.2482043,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.25937748,"width":0.18068483,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":4,"bounds":{"left":0.0,"top":0.28092578,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":5,"bounds":{"left":0.013297873,"top":0.29209897,"width":0.4644282,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Pipelines - jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.31364724,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Pipelines - jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.32482043,"width":0.039228722,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.3463687,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.3575419,"width":0.15824468,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Tab","depth":4,"bounds":{"left":0.0028257978,"top":0.38068634,"width":0.07413564,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Customize sidebar","depth":6,"bounds":{"left":0.0028257978,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Close Google Gemini (⌃X)","depth":6,"bounds":{"left":0.013796543,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Tabs from other devices","depth":6,"bounds":{"left":0.024933511,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open history (⇧⌘H)","depth":6,"bounds":{"left":0.036070477,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open bookmarks (⌘B)","depth":6,"bounds":{"left":0.04720745,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"AI Chat settings","depth":7,"bounds":{"left":0.35854387,"top":0.055067837,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Close","depth":7,"bounds":{"left":0.37051198,"top":0.055067837,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Main menu","depth":13,"bounds":{"left":0.08228058,"top":0.09736632,"width":0.015957447,"height":0.03830806},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"New chat","depth":12,"bounds":{"left":0.09823803,"top":0.10215483,"width":0.028091755,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Gemini","depth":15,"bounds":{"left":0.1008976,"top":0.10654429,"width":0.02144282,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Chat","depth":13,"bounds":{"left":0.35854387,"top":0.10215483,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Open menu for conversation actions.","depth":13,"bounds":{"left":0.37051198,"top":0.10215483,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Conversation with Gemini","depth":15,"bounds":{"left":0.079288565,"top":0.14445332,"width":0.0003324468,"height":0.0007980846},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Conversation with Gemini","depth":16,"bounds":{"left":0.079288565,"top":0.14684756,"width":0.1200133,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"You said I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.","depth":23,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Expand","depth":21,"on_screen":false,"role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a concise summary of the changes made in the code selection:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Refactoring and Dependencies","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Refactoring and Dependencies","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Replaced Interface with Implementation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Changed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusInterface","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusManager","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"in the imports and constructor dependency injection.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Redis Support:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Imported","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Illuminate\\Support\\Facades\\Redis","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to handle a newly introduced memory-logging lock mechanism.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Traits:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"WorkerIdTrait","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", along with the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"worker","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"field tracking in the final logs.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Loop and Worker Lifecycle Adjustments","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Loop and Worker Lifecycle Adjustments","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Worker Auto-Restart:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Deleted the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property, the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"constant, and the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"shouldIterateAgain()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method. The script no longer tracks execution iterations or triggers a self-managed worker restart.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Simplified Chunk Method Return:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Modified","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to always return","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Priority Handling for Throttled Entities","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Priority Handling for Throttled Entities","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Stale Record Elevation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Updated the logic in","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"rescheduleThrottledEntities()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to check if a throttled entity group is classified as \"stale\". If it is, the entities are rescheduled using","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"QueuePriorityEnum::NORMAL","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"; otherwise, they keep their respective queue priority.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Metrics, Logging, and Environment Handling","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Metrics, Logging, and Environment Handling","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Environment Tracking:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Introduced a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$environment","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property (defaulting to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"'local'","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") and a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"setEnvironment()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method to dynamically assign it.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Memory Monitoring via Datadog Gauge:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method that submits peak memory allocation to Datadog as a gauge metric (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"jiminny.reindex-memory-usage","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis Lock for Metrics Throttling:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Implemented a Redis-backed lock (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"%s-process-manager-lock","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") with a 60-second expiration inside","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to prevent over-reporting metrics.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Log Payload Update:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added total processed items (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$updatedDocuments + $deletedDocuments","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") to the final information log.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":23,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"You said are there any issues or potential bottlenecks","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"are there any issues or potential bottlenecks","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Edit","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a breakdown of the potential issues to look out for:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"1. Memory Leak Risk (OOM Crashes)","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"1. Memory Leak Risk (OOM Crashes)","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Restart Logic:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The most significant change is the removal of the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Dependency Leaks:","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.050033245,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If any underlying service (like the","depth":26,"bounds":{"left":0.17453457,"top":0.0,"width":0.08277926,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ElasticSearchClient","depth":27,"bounds":{"left":0.25930852,"top":0.0,"width":0.056848403,"height":0.015163607},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", ORM, or Datadog helper) retains references to processed entities,","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.21775267,"height":0.03631285},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"bounds":{"left":0.26811835,"top":0.0,"width":0.056848403,"height":0.015163607},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.23138298,"height":0.05546688},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Hardcoded Limits:","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.047041222,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The code still includes","depth":26,"bounds":{"left":0.17154256,"top":0.0,"width":0.05651596,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ini_set('memory_limit', '512M');","depth":27,"bounds":{"left":0.23005319,"top":0.0,"width":0.095744684,"height":0.015163607},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":". Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.23055187,"height":0.05546688},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"2. Race Condition in Redis Lock","depth":23,"bounds":{"left":0.111369684,"top":0.07940942,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"2. Race Condition in Redis Lock","depth":24,"bounds":{"left":0.111369684,"top":0.079010375,"width":0.09125665,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Non-Atomic Operations:","depth":26,"bounds":{"left":0.12450133,"top":0.10614525,"width":0.062333778,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The","depth":26,"bounds":{"left":0.18683511,"top":0.10614525,"width":0.011635638,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage","depth":27,"bounds":{"left":0.20046543,"top":0.10774142,"width":0.041888297,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method uses three distinct steps to handle the lock:","depth":26,"bounds":{"left":0.12450133,"top":0.10614525,"width":0.22722739,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"get()","depth":27,"bounds":{"left":0.14860372,"top":0.12689546,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":",","depth":26,"bounds":{"left":0.16555852,"top":0.12529927,"width":0.002493351,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"bounds":{"left":0.17004654,"top":0.12689546,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", and","depth":26,"bounds":{"left":0.18700133,"top":0.12529927,"width":0.013131649,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"expire()","depth":27,"bounds":{"left":0.20212767,"top":0.12689546,"width":0.023936171,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"bounds":{"left":0.22805852,"top":0.12529927,"width":0.0013297872,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"bounds":{"left":0.12450133,"top":0.15403032,"width":0.040059842,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If multiple workers process the same","depth":26,"bounds":{"left":0.16456117,"top":0.15403032,"width":0.09275266,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$this->updateTarget","depth":27,"bounds":{"left":0.25930852,"top":0.15562649,"width":0.056848403,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"concurrently, they can both pass the","depth":26,"bounds":{"left":0.12450133,"top":0.15403032,"width":0.22855718,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"if (! is_null(Redis::get(...)))","depth":27,"bounds":{"left":0.18351063,"top":0.17478053,"width":0.09275266,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"check at the exact same millisecond. Both will then execute","depth":26,"bounds":{"left":0.12450133,"top":0.17318435,"width":0.21575798,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"bounds":{"left":0.2122673,"top":0.19393456,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and push duplicate metrics to Datadog.","depth":26,"bounds":{"left":0.22922207,"top":0.19233839,"width":0.09823803,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Fix:","depth":26,"bounds":{"left":0.12450133,"top":0.22106944,"width":0.019281914,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"This should be replaced with a single atomic operation using Redis's","depth":26,"bounds":{"left":0.14378324,"top":0.22106944,"width":0.16821809,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"NX","depth":27,"bounds":{"left":0.31399602,"top":0.22266561,"width":0.005984043,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Not eXists) and","depth":26,"bounds":{"left":0.12450133,"top":0.22106944,"width":0.22855718,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"EX","depth":27,"bounds":{"left":0.13696809,"top":0.24181964,"width":0.005984043,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Expire) flags, or Laravel’s built-in atomic cache locks:","depth":26,"bounds":{"left":0.14494681,"top":0.24022347,"width":0.13264628,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis::set($gaugeLockName, true, 'EX', 60, 'NX')","depth":27,"bounds":{"left":0.12649602,"top":0.26097366,"width":0.14361702,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"bounds":{"left":0.27210772,"top":0.25937748,"width":0.0013297872,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"3. Infinite Loops and Caller Contract","depth":23,"bounds":{"left":0.111369684,"top":0.30925778,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"3. Infinite Loops and Caller Contract","depth":24,"bounds":{"left":0.111369684,"top":0.30885875,"width":0.10571808,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Return Value Change:","depth":26,"bounds":{"left":0.12450133,"top":0.33599362,"width":0.055352394,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"bounds":{"left":0.18301196,"top":0.33758977,"width":0.06881649,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"previously returned whether it should iterate again based on staleness or chunk size. It now always returns","depth":26,"bounds":{"left":0.12450133,"top":0.33599362,"width":0.2237367,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"bounds":{"left":0.29438165,"top":0.3567438,"width":0.011968086,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(unless the selection is totally empty).","depth":26,"bounds":{"left":0.12450133,"top":0.35514766,"width":0.2130984,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"bounds":{"left":0.12450133,"top":0.40303272,"width":0.040059842,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the code calling this method relies on a","depth":26,"bounds":{"left":0.16456117,"top":0.40303272,"width":0.10305851,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"false","depth":27,"bounds":{"left":0.26961437,"top":0.4046289,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"return to pause, sleep, or gracefully terminate, changing this to an unconditional","depth":26,"bounds":{"left":0.12450133,"top":0.40303272,"width":0.22589761,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"bounds":{"left":0.26163563,"top":0.4237829,"width":0.011968086,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"could result in an aggressive \"busy loop\" that spikes CPU usage.","depth":26,"bounds":{"left":0.12450133,"top":0.42218676,"width":0.22290559,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"4. Performance Overhead","depth":23,"bounds":{"left":0.111369684,"top":0.49122107,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"4. Performance Overhead","depth":24,"bounds":{"left":0.111369684,"top":0.49082202,"width":0.076130316,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Expensive Garbage Collection:","depth":26,"bounds":{"left":0.12450133,"top":0.5179569,"width":0.078457445,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Calling","depth":26,"bounds":{"left":0.20295878,"top":0.5179569,"width":0.019281914,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles();","depth":27,"bounds":{"left":0.22423537,"top":0.51955307,"width":0.059840426,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"on","depth":26,"bounds":{"left":0.28607047,"top":0.5179569,"width":0.00880984,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"every single chunk","depth":26,"bounds":{"left":0.29488033,"top":0.5179569,"width":0.045545213,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.","depth":26,"bounds":{"left":0.12450133,"top":0.5179569,"width":0.23420878,"height":0.05546688},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Synchronous Metric Dispatch:","depth":26,"bounds":{"left":0.12450133,"top":0.584996,"width":0.07712766,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the","depth":26,"bounds":{"left":0.20162898,"top":0.584996,"width":0.015292553,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::increment","depth":27,"bounds":{"left":0.21891622,"top":0.5865922,"width":0.053856384,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"or","depth":26,"bounds":{"left":0.27476728,"top":0.584996,"width":0.0078125,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::gauge","depth":27,"bounds":{"left":0.28457448,"top":0.5865922,"width":0.041888297,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.","depth":26,"bounds":{"left":0.12450133,"top":0.584996,"width":0.23420878,"height":0.05546688},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"5. Lack of Explicit Error Handling","depth":23,"bounds":{"left":0.111369684,"top":0.67318434,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5. Lack of Explicit Error Handling","depth":24,"bounds":{"left":0.111369684,"top":0.67278534,"width":0.094913565,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"No","depth":26,"bounds":{"left":0.12450133,"top":0.6999202,"width":0.008477394,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"try/catch","depth":27,"bounds":{"left":0.1349734,"top":0.7015164,"width":0.026928192,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Blocks:","depth":26,"bounds":{"left":0.16389628,"top":0.6999202,"width":0.019614361,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The script executes complex network operations (","depth":26,"bounds":{"left":0.18351063,"top":0.6999202,"width":0.12367021,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"loadDocuments","depth":27,"bounds":{"left":0.30917552,"top":0.7015164,"width":0.038896278,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.","depth":26,"bounds":{"left":0.12450133,"top":0.6999202,"width":0.22888963,"height":0.09377494},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":22,"bounds":{"left":0.1087101,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":22,"bounds":{"left":0.124667555,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Redo","depth":22,"bounds":{"left":0.140625,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":21,"bounds":{"left":0.15658244,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":22,"bounds":{"left":0.17253989,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":21,"bounds":{"left":0.18849733,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXTextArea","text":"Ask Gemini","depth":21,"bounds":{"left":0.14594415,"top":0.86632085,"width":0.17021276,"height":0.01915403},"on_screen":true,"value":"Ask Gemini","help_text":"","role_description":"text entry area","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Ask Gemini","depth":22,"bounds":{"left":0.14594415,"top":0.8667199,"width":0.030418882,"height":0.018355945},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Upload & tools","depth":20,"bounds":{"left":0.12732713,"top":0.8567438,"width":0.015957447,"height":0.03830806},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Microphone","depth":20,"bounds":{"left":0.32413563,"top":0.86153233,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Send message","depth":20,"bounds":{"left":0.33211437,"top":0.8631285,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":false,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.","depth":17,"bounds":{"left":0.102726065,"top":0.9173983,"width":0.21775267,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Your privacy & Gemini Opens in a new window","depth":17,"bounds":{"left":0.32047874,"top":0.9173983,"width":0.043882977,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Your privacy & Gemini","depth":18,"bounds":{"left":0.32047874,"top":0.9173983,"width":0.043882977,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Opens in a new window","depth":19,"bounds":{"left":0.079288565,"top":0.91660017,"width":0.04720745,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Summarize page","depth":7,"bounds":{"left":0.08494016,"top":0.95730245,"width":0.053523935,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Summarize page","depth":9,"bounds":{"left":0.09059176,"top":0.96249,"width":0.042220745,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Skip to content","depth":6,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Skip to content","depth":7,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Open menu","depth":10,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"Homepage (g then d)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"jiminny","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"jiminny","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"app","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"app","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Search or jump to…","depth":9,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Type","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"/","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to search","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Chat with Copilot","depth":10,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXMenuButton","text":"Open Copilot…","depth":9,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXMenuButton","text":"Create new...","depth":9,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"All issues(g then i)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"All pull requests","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"All repositories","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"You have unread notifications(g then n)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Open user navigation menu","depth":9,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Repository navigation","depth":9,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Repository navigation","depth":10,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Code","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Code","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Pull requests (32)","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Pull requests","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"32","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Agents","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Agents","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Actions","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Actions","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Wiki","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Wiki","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Security and quality","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Security and quality","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Insights","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Insights","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Settings","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Settings","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"JY-20915 add alias for EU #12105 Edit title","depth":13,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"JY-20915 add alias for EU","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"#","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"12105","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Edit title","depth":14,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Awaiting approval","depth":13,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Awaiting approval","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXMenuButton","text":"Code","depth":13,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXStaticText","text":"Code","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Open","depth":13,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"LakyLak","depth":15,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"wants to merge 5 commits into","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"master","depth":15,"on_screen":false,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"master","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"from","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915-add-domain-specific-email-text-relay","depth":16,"on_screen":false,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915-add-domain-specific-email-text-relay","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy head branch name to clipboard","depth":16,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Lines changed: 365 additions & 1 deletion","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Conversation (1)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"Conversation","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"1","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Commits (5)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Commits","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Checks (3)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Checks","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"3","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Files changed (4)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Files changed","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"4","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Open","depth":14,"bounds":{"left":0.50332445,"top":0.07222666,"width":0.011968086,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"JY-20915 add alias for EU #12105 LakyLak wants to merge 5 commits into master from JY-20915-add-domain-specific-email-text-relay Copy head branch name to clipboard","depth":14,"bounds":{"left":0.5219415,"top":0.058260176,"width":0.22922207,"height":0.042298485},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915 add alias for EU","depth":16,"bounds":{"left":0.5219415,"top":0.05865922,"width":0.05817819,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915 add alias for EU","depth":17,"bounds":{"left":0.5219415,"top":0.06304868,"width":0.05817819,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"#","depth":16,"bounds":{"left":0.5827792,"top":0.06304868,"width":0.0028257978,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"12105","depth":16,"bounds":{"left":0.585605,"top":0.06304868,"width":0.012799202,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"LakyLak","depth":18,"bounds":{"left":0.5219415,"top":0.08339984,"width":0.016123671,"height":0.011971269},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":19,"bounds":{"left":0.5219415,"top":0.08339984,"width":0.016123671,"height":0.011971269},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"wants to merge 5 commits into","depth":18,"bounds":{"left":0.539395,"top":0.08339984,"width":0.058011968,"height":0.011971269},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"master","depth":18,"bounds":{"left":0.5987367,"top":0.08180367,"width":0.018450798,"height":0.015163607},"on_screen":true,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"master","depth":19,"bounds":{"left":0.6007314,"top":0.08339984,"width":0.014461436,"height":0.011572227},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"from","depth":19,"bounds":{"left":0.6185173,"top":0.08339984,"width":0.008643617,"height":0.011971269},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915-add-domain-specific-email-text-relay","depth":19,"bounds":{"left":0.6284907,"top":0.08180367,"width":0.112034574,"height":0.015163607},"on_screen":true,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915-add-domain-specific-email-text-relay","depth":20,"bounds":{"left":0.63048536,"top":0.08339984,"width":0.10804521,"height":0.011572227},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy head branch name to clipboard","depth":19,"bounds":{"left":0.741855,"top":0.07821229,"width":0.00930851,"height":0.022346368},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Conversation","depth":12,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Conversation","depth":13,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@LakyLak","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show options","depth":15,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXHeading","text":"LakyLak commented yesterday","depth":14,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXLink","text":"LakyLak","depth":16,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"commented","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"yesterday","depth":15,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"yesterday","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"JIRA: JY-20915","depth":16,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"JIRA:","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915","depth":17,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Changes:","depth":16,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Changes:","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Support text relay for EU","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Prevent errors from EU on US","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"}]...
|
1200924818588107069
|
-1048292410045221092
|
idle
|
accessibility
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo
Share & export
Copy
Show more options
Ask Gemini
Ask Gemini
Upload & tools
Microphone
Send message
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Summarize page
Summarize page
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
All issues(g then i)
All pull requests
All repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Pull requests (32)
Pull requests
(
32
)
Agents
Agents
Actions
Actions
Wiki
Wiki
Security and quality
Security and quality
Insights
Insights
Settings
Settings
JY-20915 add alias for EU #12105 Edit title
JY-20915 add alias for EU
#
12105
Edit title
Awaiting approval
Awaiting approval
Code
Code
Open
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Lines changed: 365 additions & 1 deletion
Conversation (1)
Conversation
(
1
)
Commits (5)
Commits
(
5
)
Checks (3)
Checks
(
3
)
Files changed (4)
Files changed
(
4
)
Open
JY-20915 add alias for EU #12105 LakyLak wants to merge 5 commits into master from JY-20915-add-domain-specific-email-text-relay Copy head branch name to clipboard
JY-20915 add alias for EU
JY-20915 add alias for EU
#
12105
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Conversation
Conversation
@LakyLak
Show options
LakyLak commented yesterday
LakyLak
LakyLak
commented
yesterday
yesterday
JIRA: JY-20915
JIRA:
JY-20915
JY-20915
Changes:
Changes:
Support text relay for EU
Prevent errors from EU on US...
|
65262
|
NULL
|
NULL
|
NULL
|
|
65265
|
2289
|
6
|
2026-05-20T15:38:22.633833+00:00
|
/Users/lukas/.screenpipe/data/data/2026-05-20/1779 /Users/lukas/.screenpipe/data/data/2026-05-20/1779291502633_m1.jpg...
|
Firefox
|
JY-20915 add alias for EU by LakyLak · Pull Reques JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app — Work...
|
1
|
github.com/jiminny/app/pull/12105
|
monitor_1
|
NULL
|
NULL
|
NULL
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo
Share & export
Copy
Show more options
Ask Gemini
Ask Gemini
Upload & tools
Microphone
Send message
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Summarize page
Summarize page
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
All issues(g then i)
All pull requests
All repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Pull requests (32)
Pull requests
(
32
)
Agents
Agents
Actions
Actions
Wiki
Wiki
Security and quality
Security and quality
Insights
Insights
Settings
Settings
JY-20915 add alias for EU #12105 Edit title
JY-20915 add alias for EU
#
12105
Edit title
Awaiting approval
Awaiting approval
Code
Code
Open
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Lines changed: 365 additions & 1 deletion
Conversation (1)
Conversation
(
1
)
Commits (5)
Commits
(
5
)
Checks (3)
Checks
(
3
)
Files changed (4)
Files changed
(
4
)
Open
JY-20915 add alias for EU #12105 LakyLak wants to merge 5 commits into master from JY-20915-add-domain-specific-email-text-relay Copy head branch name to clipboard
JY-20915 add alias for EU
JY-20915 add alias for EU
#
12105
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Conversation
Conversation
@LakyLak
Show options
LakyLak commented yesterday
LakyLak
LakyLak
commented
yesterday
yesterday
JIRA: JY-20915
JIRA:
JY-20915
JY-20915
Changes:
Changes:
Support text relay for EU
Prevent errors from EU on US
Add or remove reactions
LakyLak
LakyLak
added
5
commits
yesterday
yesterday
@LakyLak
JY-20915...
|
[{"role":"AXRadioButton","text [{"role":"AXRadioButton","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Close tab","depth":5,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXRadioButton","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Pipelines - jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Pipelines - jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":4,"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":5,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Tab","depth":4,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Customize sidebar","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Close Google Gemini (⌃X)","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Tabs from other devices","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open history (⇧⌘H)","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open bookmarks (⌘B)","depth":6,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"AI Chat settings","depth":7,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Close","depth":7,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Main menu","depth":13,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"New chat","depth":12,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Gemini","depth":15,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Chat","depth":13,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Open menu for conversation actions.","depth":13,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Conversation with Gemini","depth":15,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Conversation with Gemini","depth":16,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"You said I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.","depth":23,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Expand","depth":21,"on_screen":false,"role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a concise summary of the changes made in the code selection:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Refactoring and Dependencies","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Refactoring and Dependencies","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Replaced Interface with Implementation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Changed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusInterface","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusManager","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"in the imports and constructor dependency injection.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Redis Support:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Imported","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Illuminate\\Support\\Facades\\Redis","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to handle a newly introduced memory-logging lock mechanism.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Traits:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"WorkerIdTrait","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", along with the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"worker","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"field tracking in the final logs.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Loop and Worker Lifecycle Adjustments","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Loop and Worker Lifecycle Adjustments","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Worker Auto-Restart:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Deleted the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property, the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"constant, and the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"shouldIterateAgain()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method. The script no longer tracks execution iterations or triggers a self-managed worker restart.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Simplified Chunk Method Return:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Modified","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to always return","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Priority Handling for Throttled Entities","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Priority Handling for Throttled Entities","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Stale Record Elevation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Updated the logic in","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"rescheduleThrottledEntities()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to check if a throttled entity group is classified as \"stale\". If it is, the entities are rescheduled using","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"QueuePriorityEnum::NORMAL","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"; otherwise, they keep their respective queue priority.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Metrics, Logging, and Environment Handling","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Metrics, Logging, and Environment Handling","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Environment Tracking:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Introduced a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$environment","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property (defaulting to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"'local'","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") and a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"setEnvironment()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method to dynamically assign it.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Memory Monitoring via Datadog Gauge:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method that submits peak memory allocation to Datadog as a gauge metric (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"jiminny.reindex-memory-usage","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis Lock for Metrics Throttling:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Implemented a Redis-backed lock (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"%s-process-manager-lock","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") with a 60-second expiration inside","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to prevent over-reporting metrics.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Log Payload Update:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added total processed items (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$updatedDocuments + $deletedDocuments","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") to the final information log.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":23,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"You said are there any issues or potential bottlenecks","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"are there any issues or potential bottlenecks","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Edit","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a breakdown of the potential issues to look out for:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"1. Memory Leak Risk (OOM Crashes)","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"1. Memory Leak Risk (OOM Crashes)","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Restart Logic:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The most significant change is the removal of the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Dependency Leaks:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If any underlying service (like the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ElasticSearchClient","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", ORM, or Datadog helper) retains references to processed entities,","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Hardcoded Limits:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The code still includes","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ini_set('memory_limit', '512M');","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":". Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"2. Race Condition in Redis Lock","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"2. Race Condition in Redis Lock","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Non-Atomic Operations:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method uses three distinct steps to handle the lock:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"get()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":",","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", and","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"expire()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If multiple workers process the same","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$this->updateTarget","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"concurrently, they can both pass the","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"if (! is_null(Redis::get(...)))","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"check at the exact same millisecond. Both will then execute","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and push duplicate metrics to Datadog.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Fix:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"This should be replaced with a single atomic operation using Redis's","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"NX","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Not eXists) and","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"EX","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Expire) flags, or Laravel’s built-in atomic cache locks:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis::set($gaugeLockName, true, 'EX', 60, 'NX')","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"3. Infinite Loops and Caller Contract","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"3. Infinite Loops and Caller Contract","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Return Value Change:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"previously returned whether it should iterate again based on staleness or chunk size. It now always returns","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(unless the selection is totally empty).","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the code calling this method relies on a","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"false","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"return to pause, sleep, or gracefully terminate, changing this to an unconditional","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"could result in an aggressive \"busy loop\" that spikes CPU usage.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"4. Performance Overhead","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"4. Performance Overhead","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Expensive Garbage Collection:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Calling","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles();","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"on","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"every single chunk","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Synchronous Metric Dispatch:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::increment","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"or","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::gauge","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"5. Lack of Explicit Error Handling","depth":23,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5. Lack of Explicit Error Handling","depth":24,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"No","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"try/catch","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Blocks:","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The script executes complex network operations (","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"loadDocuments","depth":27,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.","depth":26,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":22,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":22,"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Redo","depth":22,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":21,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":22,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":21,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXTextArea","text":"Ask Gemini","depth":21,"on_screen":true,"value":"Ask Gemini","help_text":"","role_description":"text entry area","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Ask Gemini","depth":22,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Upload & tools","depth":20,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Microphone","depth":20,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Send message","depth":20,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":false,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.","depth":17,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Your privacy & Gemini Opens in a new window","depth":17,"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Your privacy & Gemini","depth":18,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Opens in a new window","depth":19,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Summarize page","depth":7,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Summarize page","depth":9,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Skip to content","depth":6,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Skip to content","depth":7,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Open menu","depth":10,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"Homepage (g then d)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"jiminny","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"jiminny","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"app","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"app","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Search or jump to…","depth":9,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Type","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"/","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to search","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Chat with Copilot","depth":10,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXMenuButton","text":"Open Copilot…","depth":9,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXMenuButton","text":"Create new...","depth":9,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"All issues(g then i)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"All pull requests","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"All repositories","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"You have unread notifications(g then n)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Open user navigation menu","depth":9,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Repository navigation","depth":9,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Repository navigation","depth":10,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Code","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Code","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Pull requests (32)","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Pull requests","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"32","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Agents","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Agents","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Actions","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Actions","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Wiki","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Wiki","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Security and quality","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Security and quality","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Insights","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Insights","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Settings","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Settings","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"JY-20915 add alias for EU #12105 Edit title","depth":13,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"JY-20915 add alias for EU","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"#","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"12105","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Edit title","depth":14,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Awaiting approval","depth":13,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Awaiting approval","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXMenuButton","text":"Code","depth":13,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXStaticText","text":"Code","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Open","depth":13,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"LakyLak","depth":15,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"wants to merge 5 commits into","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"master","depth":15,"on_screen":false,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"master","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"from","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915-add-domain-specific-email-text-relay","depth":16,"on_screen":false,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915-add-domain-specific-email-text-relay","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy head branch name to clipboard","depth":16,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Lines changed: 365 additions & 1 deletion","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Conversation (1)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"Conversation","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"1","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Commits (5)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Commits","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Checks (3)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Checks","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"3","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Files changed (4)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Files changed","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"4","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Open","depth":14,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"JY-20915 add alias for EU #12105 LakyLak wants to merge 5 commits into master from JY-20915-add-domain-specific-email-text-relay Copy head branch name to clipboard","depth":14,"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915 add alias for EU","depth":16,"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915 add alias for EU","depth":17,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"#","depth":16,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"12105","depth":16,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"LakyLak","depth":18,"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":19,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"wants to merge 5 commits into","depth":18,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"master","depth":18,"on_screen":true,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"master","depth":19,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"from","depth":19,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915-add-domain-specific-email-text-relay","depth":19,"on_screen":true,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915-add-domain-specific-email-text-relay","depth":20,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy head branch name to clipboard","depth":19,"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Conversation","depth":12,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Conversation","depth":13,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@LakyLak","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show options","depth":15,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXHeading","text":"LakyLak commented yesterday","depth":14,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXLink","text":"LakyLak","depth":16,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"commented","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"yesterday","depth":15,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"yesterday","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"JIRA: JY-20915","depth":16,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"JIRA:","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915","depth":17,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Changes:","depth":16,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Changes:","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Support text relay for EU","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Prevent errors from EU on US","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Add or remove reactions","depth":16,"on_screen":false,"help_text":"","role_description":"summary","subrole":"AXSummary","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"LakyLak","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"added","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"commits","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"yesterday","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"yesterday","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@LakyLak","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"JY-20915","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false}]...
|
1223303586575050318
|
-1057299609299962088
|
idle
|
accessibility
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo
Share & export
Copy
Show more options
Ask Gemini
Ask Gemini
Upload & tools
Microphone
Send message
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Summarize page
Summarize page
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
All issues(g then i)
All pull requests
All repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Pull requests (32)
Pull requests
(
32
)
Agents
Agents
Actions
Actions
Wiki
Wiki
Security and quality
Security and quality
Insights
Insights
Settings
Settings
JY-20915 add alias for EU #12105 Edit title
JY-20915 add alias for EU
#
12105
Edit title
Awaiting approval
Awaiting approval
Code
Code
Open
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Lines changed: 365 additions & 1 deletion
Conversation (1)
Conversation
(
1
)
Commits (5)
Commits
(
5
)
Checks (3)
Checks
(
3
)
Files changed (4)
Files changed
(
4
)
Open
JY-20915 add alias for EU #12105 LakyLak wants to merge 5 commits into master from JY-20915-add-domain-specific-email-text-relay Copy head branch name to clipboard
JY-20915 add alias for EU
JY-20915 add alias for EU
#
12105
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Conversation
Conversation
@LakyLak
Show options
LakyLak commented yesterday
LakyLak
LakyLak
commented
yesterday
yesterday
JIRA: JY-20915
JIRA:
JY-20915
JY-20915
Changes:
Changes:
Support text relay for EU
Prevent errors from EU on US
Add or remove reactions
LakyLak
LakyLak
added
5
commits
yesterday
yesterday
@LakyLak
JY-20915...
|
65263
|
NULL
|
NULL
|
NULL
|
|
65266
|
2290
|
7
|
2026-05-20T15:38:35.674789+00:00
|
/Users/lukas/.screenpipe/data/data/2026-05-20/1779 /Users/lukas/.screenpipe/data/data/2026-05-20/1779291515674_m2.jpg...
|
Firefox
|
JY-20915 add alias for EU by LakyLak · Pull Reques JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app — Work...
|
1
|
github.com/jiminny/app/pull/12105
|
monitor_2
|
NULL
|
NULL
|
NULL
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo
Share & export
Copy
Show more options
Ask Gemini
Ask Gemini
Upload & tools
Microphone
Send message
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Summarize page
Summarize page
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
All issues(g then i)
All pull requests
All repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Pull requests (32)
Pull requests
(
32
)
Agents
Agents
Actions
Actions
Wiki
Wiki
Security and quality
Security and quality
Insights
Insights
Settings
Settings
JY-20915 add alias for EU #12105 Edit title
JY-20915 add alias for EU
#
12105
Edit title
Awaiting approval
Awaiting approval
Code
Code
Open
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Lines changed: 365 additions & 1 deletion
Conversation (1)
Conversation
(
1
)
Commits (5)
Commits
(
5
)
Checks (3)
Checks
(
3
)
Files changed (4)
Files changed
(
4
)
Open
JY-20915 add alias for EU #12105 LakyLak wants to merge 5 commits into master from JY-20915-add-domain-specific-email-text-relay Copy head branch name to clipboard
JY-20915 add alias for EU
JY-20915 add alias for EU
#
12105
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Conversation
Conversation
@LakyLak
Show options
LakyLak commented yesterday
LakyLak
LakyLak
commented
yesterday
yesterday
JIRA: JY-20915
JIRA:
JY-20915
JY-20915
Changes:
Changes:
Support text relay for EU
Prevent errors from EU on US
Add or remove reactions
LakyLak
LakyLak
added
5
commits
yesterday
yesterday
@LakyLak
JY-20915
JY-20915
add alias for EU
add alias for EU
8 / 10 checks OK
e5255d5
e5255d5
@LakyLak...
|
[{"role":"AXRadioButton","text [{"role":"AXRadioButton","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":4,"bounds":{"left":0.0,"top":0.0518755,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.06304868,"width":0.10106383,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":4,"bounds":{"left":0.0,"top":0.08459697,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Service-Desk - Queues - Platform team - Service space - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.09577015,"width":0.10721409,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.11731844,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.12849163,"width":0.15791224,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.15003991,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.16121309,"width":0.1796875,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.18276137,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.19393456,"width":0.12699468,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Close tab","depth":5,"bounds":{"left":0.06732048,"top":0.18994413,"width":0.007978723,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXRadioButton","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":4,"bounds":{"left":0.0,"top":0.21548285,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Coverage on New Code - app in Jiminny SonarQube Cloud","depth":5,"bounds":{"left":0.013297873,"top":0.22665602,"width":0.10139628,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":4,"bounds":{"left":0.0,"top":0.2482043,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira","depth":5,"bounds":{"left":0.013297873,"top":0.25937748,"width":0.18068483,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":4,"bounds":{"left":0.0,"top":0.28092578,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"SevenShores\\Hubspot\\Exceptions\\BadRequest: Client error: `POST https://api.hubapi.com/crm/v3/objects/contact/search` resulted in a `429 Too Many Requests` response: {\"status\":\"error\",\"message\":\"You have reached your secondly limit.\",\"errorType\":\"RATE_LIMIT","depth":5,"bounds":{"left":0.013297873,"top":0.29209897,"width":0.4644282,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Pipelines - jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.31364724,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Pipelines - jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.32482043,"width":0.039228722,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":4,"bounds":{"left":0.0,"top":0.3463687,"width":0.07962101,"height":0.032721467},"on_screen":true,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Jy 20808 schedule reset on low queue by Vasil-Jiminny · Pull Request #12106 · jiminny/app","depth":5,"bounds":{"left":0.013297873,"top":0.3575419,"width":0.15824468,"height":0.010774142},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Tab","depth":4,"bounds":{"left":0.0028257978,"top":0.38068634,"width":0.07413564,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Customize sidebar","depth":6,"bounds":{"left":0.0028257978,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Close Google Gemini (⌃X)","depth":6,"bounds":{"left":0.013796543,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Tabs from other devices","depth":6,"bounds":{"left":0.024933511,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open history (⇧⌘H)","depth":6,"bounds":{"left":0.036070477,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Open bookmarks (⌘B)","depth":6,"bounds":{"left":0.04720745,"top":0.97007185,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"AI Chat settings","depth":7,"bounds":{"left":0.35854387,"top":0.055067837,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Close","depth":7,"bounds":{"left":0.37051198,"top":0.055067837,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Main menu","depth":13,"bounds":{"left":0.08228058,"top":0.09736632,"width":0.015957447,"height":0.03830806},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"New chat","depth":12,"bounds":{"left":0.09823803,"top":0.10215483,"width":0.028091755,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Gemini","depth":15,"bounds":{"left":0.1008976,"top":0.10654429,"width":0.02144282,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"New Chat","depth":13,"bounds":{"left":0.35854387,"top":0.10215483,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Open menu for conversation actions.","depth":13,"bounds":{"left":0.37051198,"top":0.10215483,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Conversation with Gemini","depth":15,"bounds":{"left":0.079288565,"top":0.14445332,"width":0.0003324468,"height":0.0007980846},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Conversation with Gemini","depth":16,"bounds":{"left":0.079288565,"top":0.14684756,"width":0.1200133,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"You said I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\\LaravelDatadogHelper\\Datadog;7use ChaseConey\\LaravelDatadogHelper\\Datadog;8use Illuminate\\Support\\Facades\\Log;8use Illuminate\\Support\\Facades\\Log;9+use Illuminate\\Support\\Facades\\Redis;9use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\Client as ElasticSearchClient;10use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ElasticSearch\\ElasticSearchMappings;11use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\DeleteDocumentsAction;12use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\LoadDocumentsAction;13use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14use Jiminny\\Component\\ES\\Processor\\Actions\\UpsertDocumentsAction;14-use Jiminny\\Component\\ES\\Processor\\BatchStatusInterface;15+use Jiminny\\Component\\ES\\Processor\\BatchStatusManager;15use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SelectionList;16use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\DTOs\\SimpleCollection;17use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\TargetEntitiesSelector;18use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\ExtractIdsTrait;19use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\SelectEntityListTrait;20use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21use Jiminny\\Component\\ES\\Processor\\Traits\\UpdateTargetTrait;21-use Jiminny\\Component\\ES\\Processor\\Traits\\WorkerIdTrait;22use Jiminny\\Utils\\StringUtil;22use Jiminny\\Utils\\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.","depth":23,"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Expand","depth":21,"on_screen":false,"role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a concise summary of the changes made in the code selection:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Refactoring and Dependencies","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Refactoring and Dependencies","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Replaced Interface with Implementation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Changed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusInterface","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"BatchStatusManager","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"in the imports and constructor dependency injection.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Redis Support:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Imported","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Illuminate\\Support\\Facades\\Redis","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to handle a newly introduced memory-logging lock mechanism.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Traits:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"WorkerIdTrait","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", along with the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"worker","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"field tracking in the final logs.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Loop and Worker Lifecycle Adjustments","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Loop and Worker Lifecycle Adjustments","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Worker Auto-Restart:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Deleted the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property, the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"constant, and the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"shouldIterateAgain()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method. The script no longer tracks execution iterations or triggers a self-managed worker restart.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Simplified Chunk Method Return:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Modified","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to always return","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Priority Handling for Throttled Entities","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Priority Handling for Throttled Entities","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Stale Record Elevation:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Updated the logic in","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"rescheduleThrottledEntities()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to check if a throttled entity group is classified as \"stale\". If it is, the entities are rescheduled using","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"QueuePriorityEnum::NORMAL","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"; otherwise, they keep their respective queue priority.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Metrics, Logging, and Environment Handling","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Metrics, Logging, and Environment Handling","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added Environment Tracking:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Introduced a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$environment","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"property (defaulting to","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"'local'","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") and a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"setEnvironment()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method to dynamically assign it.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Memory Monitoring via Datadog Gauge:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added a","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method that submits peak memory allocation to Datadog as a gauge metric (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"jiminny.reindex-memory-usage","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis Lock for Metrics Throttling:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Implemented a Redis-backed lock (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"%s-process-manager-lock","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") with a 60-second expiration inside","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to prevent over-reporting metrics.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Log Payload Update:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Added total processed items (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$updatedDocuments + $deletedDocuments","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":") to the final information log.","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":23,"on_screen":false,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":23,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":22,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"You said are there any issues or potential bottlenecks","depth":21,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"You said","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"are there any issues or potential bottlenecks","depth":23,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy prompt","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Edit","depth":20,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Gemini said","depth":20,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Gemini said","depth":21,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Here is a breakdown of the potential issues to look out for:","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"1. Memory Leak Risk (OOM Crashes)","depth":23,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"1. Memory Leak Risk (OOM Crashes)","depth":24,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Removed Restart Logic:","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The most significant change is the removal of the","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$counter","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"RESTART_AFTER","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":").","depth":26,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Dependency Leaks:","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.050033245,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If any underlying service (like the","depth":26,"bounds":{"left":0.17453457,"top":0.0,"width":0.08277926,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ElasticSearchClient","depth":27,"bounds":{"left":0.25930852,"top":0.0,"width":0.056848403,"height":0.015163607},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", ORM, or Datadog helper) retains references to processed entities,","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.21775267,"height":0.03631285},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles()","depth":27,"bounds":{"left":0.26811835,"top":0.0,"width":0.056848403,"height":0.015163607},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.23138298,"height":0.05546688},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Hardcoded Limits:","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.047041222,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The code still includes","depth":26,"bounds":{"left":0.17154256,"top":0.0,"width":0.05651596,"height":0.01715882},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"ini_set('memory_limit', '512M');","depth":27,"bounds":{"left":0.23005319,"top":0.0,"width":0.095744684,"height":0.015163607},"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":". Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.","depth":26,"bounds":{"left":0.12450133,"top":0.0,"width":0.23055187,"height":0.05546688},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"2. Race Condition in Redis Lock","depth":23,"bounds":{"left":0.111369684,"top":0.07940942,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"2. Race Condition in Redis Lock","depth":24,"bounds":{"left":0.111369684,"top":0.079010375,"width":0.09125665,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Non-Atomic Operations:","depth":26,"bounds":{"left":0.12450133,"top":0.10614525,"width":0.062333778,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The","depth":26,"bounds":{"left":0.18683511,"top":0.10614525,"width":0.011635638,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"logMemoryUsage","depth":27,"bounds":{"left":0.20046543,"top":0.10774142,"width":0.041888297,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"method uses three distinct steps to handle the lock:","depth":26,"bounds":{"left":0.12450133,"top":0.10614525,"width":0.22722739,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"get()","depth":27,"bounds":{"left":0.14860372,"top":0.12689546,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":",","depth":26,"bounds":{"left":0.16555852,"top":0.12529927,"width":0.002493351,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"bounds":{"left":0.17004654,"top":0.12689546,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", and","depth":26,"bounds":{"left":0.18700133,"top":0.12529927,"width":0.013131649,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"expire()","depth":27,"bounds":{"left":0.20212767,"top":0.12689546,"width":0.023936171,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"bounds":{"left":0.22805852,"top":0.12529927,"width":0.0013297872,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"bounds":{"left":0.12450133,"top":0.15403032,"width":0.040059842,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If multiple workers process the same","depth":26,"bounds":{"left":0.16456117,"top":0.15403032,"width":0.09275266,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"$this->updateTarget","depth":27,"bounds":{"left":0.25930852,"top":0.15562649,"width":0.056848403,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"concurrently, they can both pass the","depth":26,"bounds":{"left":0.12450133,"top":0.15403032,"width":0.22855718,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"if (! is_null(Redis::get(...)))","depth":27,"bounds":{"left":0.18351063,"top":0.17478053,"width":0.09275266,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"check at the exact same millisecond. Both will then execute","depth":26,"bounds":{"left":0.12450133,"top":0.17318435,"width":0.21575798,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"set()","depth":27,"bounds":{"left":0.2122673,"top":0.19393456,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"and push duplicate metrics to Datadog.","depth":26,"bounds":{"left":0.22922207,"top":0.19233839,"width":0.09823803,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Fix:","depth":26,"bounds":{"left":0.12450133,"top":0.22106944,"width":0.019281914,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"This should be replaced with a single atomic operation using Redis's","depth":26,"bounds":{"left":0.14378324,"top":0.22106944,"width":0.16821809,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"NX","depth":27,"bounds":{"left":0.31399602,"top":0.22266561,"width":0.005984043,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Not eXists) and","depth":26,"bounds":{"left":0.12450133,"top":0.22106944,"width":0.22855718,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"EX","depth":27,"bounds":{"left":0.13696809,"top":0.24181964,"width":0.005984043,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(Expire) flags, or Laravel’s built-in atomic cache locks:","depth":26,"bounds":{"left":0.14494681,"top":0.24022347,"width":0.13264628,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Redis::set($gaugeLockName, true, 'EX', 60, 'NX')","depth":27,"bounds":{"left":0.12649602,"top":0.26097366,"width":0.14361702,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":".","depth":26,"bounds":{"left":0.27210772,"top":0.25937748,"width":0.0013297872,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"3. Infinite Loops and Caller Contract","depth":23,"bounds":{"left":0.111369684,"top":0.30925778,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"3. Infinite Loops and Caller Contract","depth":24,"bounds":{"left":0.111369684,"top":0.30885875,"width":0.10571808,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Return Value Change:","depth":26,"bounds":{"left":0.12450133,"top":0.33599362,"width":0.055352394,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"doUpdateEntitiesChunk()","depth":27,"bounds":{"left":0.18301196,"top":0.33758977,"width":0.06881649,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"previously returned whether it should iterate again based on staleness or chunk size. It now always returns","depth":26,"bounds":{"left":0.12450133,"top":0.33599362,"width":0.2237367,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"bounds":{"left":0.29438165,"top":0.3567438,"width":0.011968086,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(unless the selection is totally empty).","depth":26,"bounds":{"left":0.12450133,"top":0.35514766,"width":0.2130984,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The Bottleneck:","depth":26,"bounds":{"left":0.12450133,"top":0.40303272,"width":0.040059842,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the code calling this method relies on a","depth":26,"bounds":{"left":0.16456117,"top":0.40303272,"width":0.10305851,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"false","depth":27,"bounds":{"left":0.26961437,"top":0.4046289,"width":0.014960106,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"return to pause, sleep, or gracefully terminate, changing this to an unconditional","depth":26,"bounds":{"left":0.12450133,"top":0.40303272,"width":0.22589761,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"true","depth":27,"bounds":{"left":0.26163563,"top":0.4237829,"width":0.011968086,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"could result in an aggressive \"busy loop\" that spikes CPU usage.","depth":26,"bounds":{"left":0.12450133,"top":0.42218676,"width":0.22290559,"height":0.03631285},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"4. Performance Overhead","depth":23,"bounds":{"left":0.111369684,"top":0.49122107,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"4. Performance Overhead","depth":24,"bounds":{"left":0.111369684,"top":0.49082202,"width":0.076130316,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Expensive Garbage Collection:","depth":26,"bounds":{"left":0.12450133,"top":0.5179569,"width":0.078457445,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Calling","depth":26,"bounds":{"left":0.20295878,"top":0.5179569,"width":0.019281914,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"gc_collect_cycles();","depth":27,"bounds":{"left":0.22423537,"top":0.51955307,"width":0.059840426,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"on","depth":26,"bounds":{"left":0.28607047,"top":0.5179569,"width":0.00880984,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"every single chunk","depth":26,"bounds":{"left":0.29488033,"top":0.5179569,"width":0.045545213,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.","depth":26,"bounds":{"left":0.12450133,"top":0.5179569,"width":0.23420878,"height":0.05546688},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Synchronous Metric Dispatch:","depth":26,"bounds":{"left":0.12450133,"top":0.584996,"width":0.07712766,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"If the","depth":26,"bounds":{"left":0.20162898,"top":0.584996,"width":0.015292553,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::increment","depth":27,"bounds":{"left":0.21891622,"top":0.5865922,"width":0.053856384,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"or","depth":26,"bounds":{"left":0.27476728,"top":0.584996,"width":0.0078125,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Datadog::gauge","depth":27,"bounds":{"left":0.28457448,"top":0.5865922,"width":0.041888297,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.","depth":26,"bounds":{"left":0.12450133,"top":0.584996,"width":0.23420878,"height":0.05546688},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"5. Lack of Explicit Error Handling","depth":23,"bounds":{"left":0.111369684,"top":0.67318434,"width":0.24734043,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5. Lack of Explicit Error Handling","depth":24,"bounds":{"left":0.111369684,"top":0.67278534,"width":0.094913565,"height":0.020351157},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"No","depth":26,"bounds":{"left":0.12450133,"top":0.6999202,"width":0.008477394,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"try/catch","depth":27,"bounds":{"left":0.1349734,"top":0.7015164,"width":0.026928192,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Blocks:","depth":26,"bounds":{"left":0.16389628,"top":0.6999202,"width":0.019614361,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"The script executes complex network operations (","depth":26,"bounds":{"left":0.18351063,"top":0.6999202,"width":0.12367021,"height":0.01715882},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"loadDocuments","depth":27,"bounds":{"left":0.30917552,"top":0.7015164,"width":0.038896278,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":", Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.","depth":26,"bounds":{"left":0.12450133,"top":0.6999202,"width":0.22888963,"height":0.09377494},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXCheckBox","text":"Good response","depth":22,"bounds":{"left":0.1087101,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXCheckBox","text":"Bad response","depth":22,"bounds":{"left":0.124667555,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"toggle button","subrole":"AXToggle","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Redo","depth":22,"bounds":{"left":0.140625,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Share & export","depth":21,"bounds":{"left":0.15658244,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Copy","depth":22,"bounds":{"left":0.17253989,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show more options","depth":21,"bounds":{"left":0.18849733,"top":0.80885875,"width":0.011968086,"height":0.028731046},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXTextArea","text":"Ask Gemini","depth":21,"bounds":{"left":0.14594415,"top":0.86632085,"width":0.17021276,"height":0.01915403},"on_screen":true,"value":"Ask Gemini","help_text":"","role_description":"text entry area","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Ask Gemini","depth":22,"bounds":{"left":0.14594415,"top":0.8667199,"width":0.030418882,"height":0.018355945},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Upload & tools","depth":20,"bounds":{"left":0.12732713,"top":0.8567438,"width":0.015957447,"height":0.03830806},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXButton","text":"Microphone","depth":20,"bounds":{"left":0.32413563,"top":0.86153233,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Send message","depth":20,"bounds":{"left":0.33211437,"top":0.8631285,"width":0.010638298,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":false,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.","depth":17,"bounds":{"left":0.102726065,"top":0.9173983,"width":0.21775267,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Your privacy & Gemini Opens in a new window","depth":17,"bounds":{"left":0.32047874,"top":0.9173983,"width":0.043882977,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Your privacy & Gemini","depth":18,"bounds":{"left":0.32047874,"top":0.9173983,"width":0.043882977,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Opens in a new window","depth":19,"bounds":{"left":0.079288565,"top":0.91660017,"width":0.04720745,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Summarize page","depth":7,"bounds":{"left":0.08494016,"top":0.95730245,"width":0.053523935,"height":0.025538707},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Summarize page","depth":9,"bounds":{"left":0.09059176,"top":0.96249,"width":0.042220745,"height":0.015163607},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Skip to content","depth":6,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Skip to content","depth":7,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Open menu","depth":10,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"Homepage (g then d)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"jiminny","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"jiminny","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"app","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"app","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Search or jump to…","depth":9,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Type","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"/","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"to search","depth":12,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Chat with Copilot","depth":10,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXMenuButton","text":"Open Copilot…","depth":9,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXMenuButton","text":"Create new...","depth":9,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"All issues(g then i)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"All pull requests","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"All repositories","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"You have unread notifications(g then n)","depth":9,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Open user navigation menu","depth":9,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Repository navigation","depth":9,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Repository navigation","depth":10,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Code","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Code","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Pull requests (32)","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Pull requests","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"32","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Agents","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Agents","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Actions","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Actions","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Wiki","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Wiki","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Security and quality","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Security and quality","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Insights","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Insights","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"Settings","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Settings","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"JY-20915 add alias for EU #12105 Edit title","depth":13,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"JY-20915 add alias for EU","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"#","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"12105","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Edit title","depth":14,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Awaiting approval","depth":13,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Awaiting approval","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXMenuButton","text":"Code","depth":13,"on_screen":false,"help_text":"","role_description":"menu button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXStaticText","text":"Code","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Open","depth":13,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"LakyLak","depth":15,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"wants to merge 5 commits into","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"master","depth":15,"on_screen":false,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"master","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"from","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915-add-domain-specific-email-text-relay","depth":16,"on_screen":false,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915-add-domain-specific-email-text-relay","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy head branch name to clipboard","depth":16,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Lines changed: 365 additions & 1 deletion","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Conversation (1)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":true},{"role":"AXStaticText","text":"Conversation","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"1","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Commits (5)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Commits","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Checks (3)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Checks","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"3","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXRadioButton","text":"Files changed (4)","depth":16,"on_screen":false,"help_text":"","role_description":"tab","subrole":"AXTabButton","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"Files changed","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"(","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"4","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":")","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Open","depth":14,"bounds":{"left":0.50332445,"top":0.07222666,"width":0.011968086,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"JY-20915 add alias for EU #12105 LakyLak wants to merge 5 commits into master from JY-20915-add-domain-specific-email-text-relay Copy head branch name to clipboard","depth":14,"bounds":{"left":0.5219415,"top":0.058260176,"width":0.22922207,"height":0.042298485},"on_screen":true,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915 add alias for EU","depth":16,"bounds":{"left":0.5219415,"top":0.05865922,"width":0.05817819,"height":0.01915403},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915 add alias for EU","depth":17,"bounds":{"left":0.5219415,"top":0.06304868,"width":0.05817819,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"#","depth":16,"bounds":{"left":0.5827792,"top":0.06304868,"width":0.0028257978,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"12105","depth":16,"bounds":{"left":0.585605,"top":0.06304868,"width":0.012799202,"height":0.013567438},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"LakyLak","depth":18,"bounds":{"left":0.5219415,"top":0.08339984,"width":0.016123671,"height":0.011971269},"on_screen":true,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":19,"bounds":{"left":0.5219415,"top":0.08339984,"width":0.016123671,"height":0.011971269},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"wants to merge 5 commits into","depth":18,"bounds":{"left":0.539395,"top":0.08339984,"width":0.058011968,"height":0.011971269},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"master","depth":18,"bounds":{"left":0.5987367,"top":0.08180367,"width":0.018450798,"height":0.015163607},"on_screen":true,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"master","depth":19,"bounds":{"left":0.6007314,"top":0.08339984,"width":0.014461436,"height":0.011572227},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"from","depth":19,"bounds":{"left":0.6185173,"top":0.08339984,"width":0.008643617,"height":0.011971269},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915-add-domain-specific-email-text-relay","depth":19,"bounds":{"left":0.6284907,"top":0.08180367,"width":0.112034574,"height":0.015163607},"on_screen":true,"role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915-add-domain-specific-email-text-relay","depth":20,"bounds":{"left":0.63048536,"top":0.08339984,"width":0.10804521,"height":0.011572227},"on_screen":true,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Copy head branch name to clipboard","depth":19,"bounds":{"left":0.741855,"top":0.07821229,"width":0.00930851,"height":0.022346368},"on_screen":true,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXHeading","text":"Conversation","depth":12,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Conversation","depth":13,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@LakyLak","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXButton","text":"Show options","depth":15,"on_screen":false,"help_text":"","role_description":"button","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXHeading","text":"LakyLak commented yesterday","depth":14,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXLink","text":"LakyLak","depth":16,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"commented","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"yesterday","depth":15,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"yesterday","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"JIRA: JY-20915","depth":16,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"JIRA:","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"JY-20915","depth":17,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXHeading","text":"Changes:","depth":16,"on_screen":false,"help_text":"","role_description":"heading","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Changes:","depth":17,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Support text relay for EU","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"Prevent errors from EU on US","depth":18,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"Add or remove reactions","depth":16,"on_screen":false,"help_text":"","role_description":"summary","subrole":"AXSummary","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"LakyLak","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"LakyLak","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"added","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"5","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXStaticText","text":"commits","depth":14,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"yesterday","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"yesterday","depth":16,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@LakyLak","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXLink","text":"JY-20915","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"JY-20915","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"add alias for EU","depth":14,"on_screen":false,"help_text":"JY-20915 add alias for EU","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"add alias for EU","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXButton","text":"8 / 10 checks OK","depth":14,"on_screen":false,"help_text":"","role_description":"summary","subrole":"AXSummary","is_enabled":true,"is_focused":false,"is_selected":false,"is_expanded":false},{"role":"AXLink","text":"e5255d5","depth":14,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false},{"role":"AXStaticText","text":"e5255d5","depth":15,"on_screen":false,"help_text":"","role_description":"text","subrole":"AXUnknown"},{"role":"AXLink","text":"@LakyLak","depth":12,"on_screen":false,"help_text":"","role_description":"link","subrole":"AXUnknown","is_enabled":true,"is_focused":false,"is_selected":false}]...
|
2341065158970747303
|
-1048292959795792100
|
idle
|
accessibility
|
NULL
|
Platform Sprint 4 Q2 - Platform Team - Scrum Board Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 4 Q2 - Platform Team - Scrum Board - Jira
Service-Desk - Queues - Platform team - Service space - Jira
Service-Desk - Queues - Platform team - Service space - Jira
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
JY-20676 delete AJ reports related objects by LakyLak · Pull Request #12098 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
Allow owner's role to be selected when setting up a trial by LakyLak · Pull Request #12092 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
JY-20915 add alias for EU by LakyLak · Pull Request #12105 · jiminny/app
Close tab
Coverage on New Code - app in Jiminny SonarQube Cloud
Coverage on New Code - app in Jiminny SonarQube Cloud
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
[JY-20915] Add environment-specific email domains for text relay to prevent duplicate processing - Jira
SevenShores\Hubspot\Exceptions\BadRequest: Client error: `POST [URL_WITH_CREDENTIALS] -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected. Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
You said
I’m on page “<tabTitle>Jy 20808 schedule reset on low queue by Vasil-Jimi</tabTitle>” with “<selection>@@ -6,38 +6,35 @@667use ChaseConey\LaravelDatadogHelper\Datadog;7use ChaseConey\LaravelDatadogHelper\Datadog;8use Illuminate\Support\Facades\Log;8use Illuminate\Support\Facades\Log;9+use Illuminate\Support\Facades\Redis;9use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\Client as ElasticSearchClient;10use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ElasticSearch\ElasticSearchMappings;11use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\DeleteDocumentsAction;12use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\LoadDocumentsAction;13use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14use Jiminny\Component\ES\Processor\Actions\UpsertDocumentsAction;14-use Jiminny\Component\ES\Processor\BatchStatusInterface;15+use Jiminny\Component\ES\Processor\BatchStatusManager;15use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SelectionList;16use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\DTOs\SimpleCollection;17use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\TargetEntitiesSelector;18use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\ExtractIdsTrait;19use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\SelectEntityListTrait;20use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21use Jiminny\Component\ES\Processor\Traits\UpdateTargetTrait;21-use Jiminny\Component\ES\Processor\Traits\WorkerIdTrait;22use Jiminny\Utils\StringUtil;22use Jiminny\Utils\StringUtil;232324class UpdateProcessManager24class UpdateProcessManager25{25{26use ExtractIdsTrait;26use ExtractIdsTrait;27use SelectEntityListTrait;27use SelectEntityListTrait;28use UpdateTargetTrait;28use UpdateTargetTrait;29-use WorkerIdTrait;302931-/**30+private const string GAUGE_LOCK_KEY = '%s-process-manager-lock';32- * Restart the script every X iterations, to allow the OS to collect any unused memory31+33- */32+private string $environment = 'local';34-private const int RESTART_AFTER = 20; // Iterations35-private int $counter = 0;363337public function __construct(34public function __construct(38private readonly ElasticSearchClient $esClient,35private readonly ElasticSearchClient $esClient,39private readonly TargetEntitiesSelector $entitySelector,36private readonly TargetEntitiesSelector $entitySelector,40-private readonly BatchStatusInterface $batchStatusManager,37+private readonly BatchStatusManager $batchStatusManager,41private readonly DeleteDocumentsAction $deleteDocumentsAction,38private readonly DeleteDocumentsAction $deleteDocumentsAction,42private readonly UpsertDocumentsAction $upsertDocumentsAction,39private readonly UpsertDocumentsAction $upsertDocumentsAction,43private readonly LoadDocumentsAction $loadDocumentsAction,40private readonly LoadDocumentsAction $loadDocumentsAction,@@ -49,6 +46,11 @@ public function __construct(49ini_set('memory_limit', '512M');46ini_set('memory_limit', '512M');50 }47 }514849+public function setEnvironment(string $environment): void50+ {51+$this->environment = $environment;52+ }53+52public function bootstrap(): void54public function bootstrap(): void53 {55 {54// Prepare the batch status manager56// Prepare the batch status manager@@ -73,21 +75,15 @@ public function bootstrap(): void737574public function doUpdateEntitiesChunk(): bool76public function doUpdateEntitiesChunk(): bool75 {77 {76-$batchName = $this->batchStatusManager->generateName();77$startTime = microtime(true);78$startTime = microtime(true);79+$batchName = $this->batchStatusManager->generateName();788079/** Select Entity IDs for processing */81/** Select Entity IDs for processing */80$selectionList = $this->entitySelector->select();82$selectionList = $this->entitySelector->select();81if ($selectionList->isEmpty()) {83if ($selectionList->isEmpty()) {82return false;84return false;83 }85 }848685-/**86- * A stale chunk will always be smaller than a regular chunk.87- * That doesn't mean there aren't any other scheduled entities for processing88- */89-$shouldIterateAgain = $this->shouldIterateAgain($selectionList);90-91/** Mark the batch as scheduled for work, then load the actual documents */87/** Mark the batch as scheduled for work, then load the actual documents */92$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);88$this->batchStatusManager->scheduleWork($selectionList->getAllIds(), $batchName);93$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());89$documentLoad = $this->loadDocumentsAction->loadDocuments($this->updateTarget, $selectionList->getAllIds());@@ -103,42 +99,23 @@ public function doUpdateEntitiesChunk(): bool103 unset($selectionList);99 unset($selectionList);104gc_collect_cycles();100gc_collect_cycles();105101106-/**102+/** Write stats and logs */107- * Write stats and logs108- */109 Datadog::increment(103 Datadog::increment(110 stats: 'jiminny.es-documents-synced',104 stats: 'jiminny.es-documents-synced',111 sampleRate: 1,105 sampleRate: 1,112 value: $updatedDocuments + $deletedDocuments106 value: $updatedDocuments + $deletedDocuments113 );107 );114108109+$this->logMemoryUsage();110+115 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [111 Log::info('[ EsUpdateProcessManager ] Finished updating entities in ES', [116-'worker' => $this->workerId,117'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),112'peak_memory' => StringUtil::filesizeToHuman(memory_get_peak_usage(true)),118'elapsed_seconds' => round(microtime(true) - $startTime, 2),113'elapsed_seconds' => round(microtime(true) - $startTime, 2),119'update_target' => $this->updateTarget,114'update_target' => $this->updateTarget,120-'should_iterate_again' => $shouldIterateAgain,115+'processed' => $updatedDocuments + $deletedDocuments,121 ]);116 ]);122117123-$this->counter++;118+return true;124-125-return $shouldIterateAgain;126- }127-128-private function shouldIterateAgain(SelectionList $selectionList): bool129- {130-if ($selectionList->isStaleChunkSelected() || $selectionList->isFullChunk()) {131-/**132- * Restart the worker to allow the OS to collect any unused memory133- */134-if ($this->counter >= self::RESTART_AFTER) {135-return false;136- }137-138-return true;139- }140-141-return false;142 }119 }143120144private function executeDelete(SimpleCollection $documentsToDelete): int121private function executeDelete(SimpleCollection $documentsToDelete): int@@ -162,13 +139,41 @@ private function rescheduleThrottledEntities(SelectionList $selectionList): void162$groups = $selectionList->getEntityListGroups();139$groups = $selectionList->getEntityListGroups();163foreach ($groups as $eachGroup) {140foreach ($groups as $eachGroup) {164$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);141$groupEntityIds = $selectionList->getEntityListItemsByGroup($eachGroup);142+$groupPriority = QueuePriorityEnum::tryFromString($eachGroup);143+165if (! empty($groupEntityIds)) {144if (! empty($groupEntityIds)) {145+/**146+ * Throttled stale records are rescheduled as normal priority147+ * all other records are rescheduled to their respective queue148+ */166 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(149 AsyncUpdateElasticSearch::bulkMarkEntitiesForUpdate(167 entityType: $this->updateTarget,150 entityType: $this->updateTarget,168 entityIds: $groupEntityIds,151 entityIds: $groupEntityIds,169- priority: QueuePriorityEnum::tryFromString($eachGroup)152+ priority: $groupPriority->isStale() ? QueuePriorityEnum::NORMAL : $groupPriority,170 );153 );171 }154 }172 }155 }173 }156 }157+158+private function logMemoryUsage(): void159+ {160+$gaugeLockName = sprintf(self::GAUGE_LOCK_KEY, $this->updateTarget);161+if (! is_null(Redis::get($gaugeLockName))) {162+return;163+ }164+165+ Redis::set($gaugeLockName, true);166+ Redis::expire($gaugeLockName, 60);167+168+// Add memory usage of the process as well169+ Datadog::gauge(170+ stat: 'jiminny.reindex-memory-usage',171+ value: memory_get_peak_usage(true),172+ sampleRate: 1,173+ tags: [174+'environment' => $this->environment,175+'type' => $this->updateTarget,176+ ]177+ );178+ }174}179}</selection>” selected.
Please summarize the selection using precise and concise language. Use headers and bulleted lists in the summary, to make it scannable. Maintain the meaning and factual accuracy.
Expand
Copy prompt
Gemini said
Gemini said
Here is a concise summary of the changes made in the code selection:
Refactoring and Dependencies
Refactoring and Dependencies
Replaced Interface with Implementation:
Changed
BatchStatusInterface
to
BatchStatusManager
in the imports and constructor dependency injection.
Added Redis Support:
Imported
Illuminate\Support\Facades\Redis
to handle a newly introduced memory-logging lock mechanism.
Removed Traits:
Removed
WorkerIdTrait
, along with the
worker
field tracking in the final logs.
Loop and Worker Lifecycle Adjustments
Loop and Worker Lifecycle Adjustments
Removed Worker Auto-Restart:
Deleted the
$counter
property, the
RESTART_AFTER
constant, and the
shouldIterateAgain()
method. The script no longer tracks execution iterations or triggers a self-managed worker restart.
Simplified Chunk Method Return:
Modified
doUpdateEntitiesChunk()
to always return
true
upon completion, rather than determining whether to iterate again based on chunk fullness or staleness.
Priority Handling for Throttled Entities
Priority Handling for Throttled Entities
Stale Record Elevation:
Updated the logic in
rescheduleThrottledEntities()
to check if a throttled entity group is classified as "stale". If it is, the entities are rescheduled using
QueuePriorityEnum::NORMAL
; otherwise, they keep their respective queue priority.
Metrics, Logging, and Environment Handling
Metrics, Logging, and Environment Handling
Added Environment Tracking:
Introduced a
$environment
property (defaulting to
'local'
) and a
setEnvironment()
method to dynamically assign it.
Memory Monitoring via Datadog Gauge:
Added a
logMemoryUsage()
method that submits peak memory allocation to Datadog as a gauge metric (
jiminny.reindex-memory-usage
).
Redis Lock for Metrics Throttling:
Implemented a Redis-backed lock (
%s-process-manager-lock
) with a 60-second expiration inside
logMemoryUsage()
to prevent over-reporting metrics.
Log Payload Update:
Added total processed items (
$updatedDocuments + $deletedDocuments
) to the final information log.
Good response
Bad response
Share & export
Copy
Show more options
You said are there any issues or potential bottlenecks
You said
are there any issues or potential bottlenecks
Copy prompt
Edit
Gemini said
Gemini said
Based on the code selection provided, there are several potential issues, bottlenecks, and architectural risks—primarily revolving around memory management, race conditions, and error handling.
Here is a breakdown of the potential issues to look out for:
1. Memory Leak Risk (OOM Crashes)
1. Memory Leak Risk (OOM Crashes)
Removed Restart Logic:
The most significant change is the removal of the
$counter
and
RESTART_AFTER
logic. Previously, the worker restarted itself to let the OS reclaim memory. Now, it relies entirely on PHP's internal garbage collection (
gc_collect_cycles()
).
Dependency Leaks:
If any underlying service (like the
ElasticSearchClient
, ORM, or Datadog helper) retains references to processed entities,
gc_collect_cycles()
will not be able to free them. The worker will eventually hit the memory limit and crash with a fatal Out Of Memory (OOM) error.
Hardcoded Limits:
The code still includes
ini_set('memory_limit', '512M');
. Hardcoding environment constraints inside application logic makes it difficult to scale or tune workers dynamically via infrastructure configurations.
2. Race Condition in Redis Lock
2. Race Condition in Redis Lock
Non-Atomic Operations:
The
logMemoryUsage
method uses three distinct steps to handle the lock:
get()
,
set()
, and
expire()
.
The Bottleneck:
If multiple workers process the same
$this->updateTarget
concurrently, they can both pass the
if (! is_null(Redis::get(...)))
check at the exact same millisecond. Both will then execute
set()
and push duplicate metrics to Datadog.
The Fix:
This should be replaced with a single atomic operation using Redis's
NX
(Not eXists) and
EX
(Expire) flags, or Laravel’s built-in atomic cache locks:
Redis::set($gaugeLockName, true, 'EX', 60, 'NX')
.
3. Infinite Loops and Caller Contract
3. Infinite Loops and Caller Contract
Return Value Change:
doUpdateEntitiesChunk()
previously returned whether it should iterate again based on staleness or chunk size. It now always returns
true
(unless the selection is totally empty).
The Bottleneck:
If the code calling this method relies on a
false
return to pause, sleep, or gracefully terminate, changing this to an unconditional
true
could result in an aggressive "busy loop" that spikes CPU usage.
4. Performance Overhead
4. Performance Overhead
Expensive Garbage Collection:
Calling
gc_collect_cycles();
on
every single chunk
can be CPU-intensive. While it is necessary since the restart logic was removed, it adds overhead to the loop.
Synchronous Metric Dispatch:
If the
Datadog::increment
or
Datadog::gauge
methods are making synchronous API calls over HTTP or TCP (rather than non-blocking UDP like standard StatsD), this will block the worker and significantly slow down the indexing process.
5. Lack of Explicit Error Handling
5. Lack of Explicit Error Handling
No
try/catch
Blocks:
The script executes complex network operations (
loadDocuments
, Datadog logging, ElasticSearch updates/deletes) without visible exception handling in this snippet. If a single document fails to load or the Elasticsearch cluster drops a connection, the unhandled exception will crash the worker mid-batch, potentially leaving the batch in an inconsistent state.
Good response
Bad response
Redo
Share & export
Copy
Show more options
Ask Gemini
Ask Gemini
Upload & tools
Microphone
Send message
Your Jiminny chats aren’t used to improve our models. Gemini is AI. It can make mistakes, so double check it.
Your privacy & Gemini Opens in a new window
Your privacy & Gemini
Opens in a new window
Summarize page
Summarize page
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
app
app
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
All issues(g then i)
All pull requests
All repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Pull requests (32)
Pull requests
(
32
)
Agents
Agents
Actions
Actions
Wiki
Wiki
Security and quality
Security and quality
Insights
Insights
Settings
Settings
JY-20915 add alias for EU #12105 Edit title
JY-20915 add alias for EU
#
12105
Edit title
Awaiting approval
Awaiting approval
Code
Code
Open
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Lines changed: 365 additions & 1 deletion
Conversation (1)
Conversation
(
1
)
Commits (5)
Commits
(
5
)
Checks (3)
Checks
(
3
)
Files changed (4)
Files changed
(
4
)
Open
JY-20915 add alias for EU #12105 LakyLak wants to merge 5 commits into master from JY-20915-add-domain-specific-email-text-relay Copy head branch name to clipboard
JY-20915 add alias for EU
JY-20915 add alias for EU
#
12105
LakyLak
LakyLak
wants to merge 5 commits into
master
master
from
JY-20915-add-domain-specific-email-text-relay
JY-20915-add-domain-specific-email-text-relay
Copy head branch name to clipboard
Conversation
Conversation
@LakyLak
Show options
LakyLak commented yesterday
LakyLak
LakyLak
commented
yesterday
yesterday
JIRA: JY-20915
JIRA:
JY-20915
JY-20915
Changes:
Changes:
Support text relay for EU
Prevent errors from EU on US
Add or remove reactions
LakyLak
LakyLak
added
5
commits
yesterday
yesterday
@LakyLak
JY-20915
JY-20915
add alias for EU
add alias for EU
8 / 10 checks OK
e5255d5
e5255d5
@LakyLak...
|
NULL
|
NULL
|
NULL
|
NULL
|