|
Nikolay Yankov (Presenting)
Nikolay Yankov (Presen Nikolay Yankov (Presenting)
Nikolay Yankov (Presenting)
People
6
Take notes with Gemini
Take notes with Gemini
Gemini
Gemini
Unpin Nikolay Yankov's presentation from your main screen
You can't unmute someone else's presentation
More options for Nikolay Yankov
Zoom in
Open in new window
Enter Full Screen
Pin Stefka Stoyanova to your main screen
You can't unmute someone else
More options for Stefka Stoyanova
Stefka Stoyanova
Pin Nikolay Yankov to your main screen
Mute Nikolay Yankov's microphone
More options for Nikolay Yankov
Nikolay Yankov
Pin Steliyan Georgiev to your main screen
Mute Steliyan Georgiev's microphone
More options for Steliyan Georgiev
Steliyan Georgiev
Pin Nikolay Ivanov to your main screen
Mute Nikolay Ivanov's microphone
More options for Nikolay Ivanov
Nikolay Ivanov
You’re continuously framed
Backgrounds and effects
More options for Lukas Kovalik
Lukas Kovalik
Others might see more of your background. Click to view your full video.
9:48
AM
Daily - Platform
Daily - Platform
Audio settings
Turn off microphone
Video settings
Turn off camera
Nikolay Yankov is presenting
Send a reaction
Turn on captions
Raise hand (ctrl + ⌘ + h)
More options
Leave call
Meeting details
Chat with everyone
Meeting tools
Your microphone is on.
meet.google.com / Daily - Platform
FirefoxFileEditPlatform Sprint 1 Q2 - Platform Tea@ Jy 19798 evaluation for ai activ+ New labHistoryBookmarksProfilesToolsWindow Held• github.com/jiminny/prophet/pull/468/changesjiminny / prophet &‹> CodeO Issues I? Pull requests 6& Agents• Actions# Projects DD Wiki © Security and quality 20l Insightsộs Settings® On April 24 we'll start using GitHub Copilot interaction data for Al model training unless you opt out. Review this update and manage your preferences in your GitHub account settings.Jy 19798 evaluation for ai activity types #468 °I1 Opennikolaybiaivanov wants to merge 9 commits into master from JY-19798-evaluation-for-ai-activity-typesConversation 3a Commits 9E ChecksI• All commits +Q Filter files....• E eval_output/ai_activity_type/e….• E ground_truthĐ gt_-E4rcp0Bd4RvtuaTN2RY...D gt_7067cZ0Bd4RvtuaT1GT...Đ gt_7k6scZ0Bd4RvtuaT9GQ...• gt_807ncZOBd4RvtuaTumT...9t_8E6-cZOBd4RvtuaThmS...Đ gt_8k7mcZOBd4RvtuaTO2R...Đ gt_8U7VcZ0Bd4RvtuaTOW...D gt_904LcpOBd4RvtuaTyGT...• gt_9E7ycZ0Bd4RvtuaTDGR..D 9t_9k4Ccp0Bd4RvtuaToGTi...Đ 9t_9U4Ccp0Bd4RvtuaT|2R...Đ gt_D1ChcZ0BCSsQHiSAHz…D gt_E1C9cZOBCSSQHiSAtTb...• gt_EFCjcZOBCSsQHiSAQTa...D 9t_EIC7cZOBCSsQHiSAozY...Đ gt_EVC1cZOBCSsQHiSAKja…Đ gt_F1DscZ0BCSsQHiSAmT..Đ gt_FFDKCZOBCSsQHiSACja...• gt_FIDTcZ0BCSsQHiSA_TY…|Đ 9t_FVDKcZOBCSsQHiSAaD...Đ 9t_G1AZCpOBCSSQHiSAHz...D gt_G80fcZ0BahljYMXOtT03...Đ gt_g8SgcZOBRbelYNJH50...• gt_GFD9cZOBCSsQHiSAHz....Đ 9t_GIAGcpOBCSsQHiSAkza...Đ gt_gsSScZOBRbelYNJHvuD...Đ gt_GVD-cZOBCSsQHiSAUz...Files changed 203® This page has been optimized for large pull requests. Some browser features (like Find on Page or Select All) may not work as expected.Y eval_output/ai_activity_type/eval_datasets/v2-from-logs-replayable/ground_truth/gt_-E4rcp0Bd4RvtuaTN2RY.json0a -0,0 +1,4eval_output/ai_activity_type/eval_datasets/v2-from-logs-replayable/ground_truth/gt_7067cZ0Bd4RvtuaT1GTC.json-0,0 +1,4 @ev eval output/ai activity type/eval datasets/v2-from-loqs-replayable/ground truth/qt 7k6scZ0Bd4RvtuaT9GQV.ison [L•a -0,0 +1,4 caV eval output/ai activity tvpe/eval datasets/v2-from-loas-replavable/around truth/at 807ncZ0Bd4RvtuaTumTs. ison fL∞a -0,0 +1,4 ca‹ >0 lul • | Daily - Platform • now100% C 8• Tue 14 Apr 9:48:43Q Type to search• Awaiting approvalCode -Preview) -+25,671 m0000• 0 / 203 viewedSubmit review -Switch to single file mode+4 Uuuud_ Viewed2+ {+"ai_activity_type_reasoning": "Both Connectd rep (Jake Chweidan) and prospect (Andrew Thomas) participated and callduration is 33+ minutes, exceeding 5 minutes; therefore the criteria for 'Meeting Sat' are met.",3+ ÷°"ai_activity_type": "Meeting Sat"+4 UUUUU ]Viewed ••2"ai_activity_type_reasoning": "Vendor rep Toby Firkins (Connectd) and prospect Chris Lockyer both attended, and callduration is 12 minutes 11 seconds (>5 minutes), satisfying both criteria.","ai_activity_type": "Meeting Sat"+400000 _ Viewed |+ {"ai_activity_type_reasoning": "Both a Connectd rep (Alex Pertwee) and the prospect (Dan Hyde) attended the call, andthe call duration was 33 minutes 41 seconds, which is longer than 5 minutes\u2014meeting the criteria.","ai_activity_type": "Meeting Sat"+ 3+4 .0000_ Viewed+ {"ai_activity_type_reasoning": "Call metadata shows an SDR (Mads Hartmann Rasmussen) from Velatir calling Entech MarineA/S for 1m23s, typical of a short outbound cold call to book a demo. No transcript content indicating a demo, follow-up,or onboarding. The description matches an outbound cold call to book a demo.","ai_activity_type": "Outbound Phone Call"+ }...
|
Dia
|
Work: Meet - Daily - Pla…
|
NULL
|
8487
|
|
DiaFileEditViewTabsBookmarksHistoryExtensionsWindo DiaFileEditViewTabsBookmarksHistoryExtensionsWindowHelpmeet.google.com / Daily - PlatformNikolay Yankov (Presenting)(abl| Daily - Platform • now100% <78• Tue 14 Apr 9:48:44Chat68•Moorteh3y19798 eval(7 Service-DeskJr Foet AmesorCost | Claude@ Actions secrC C DependatE SSHDatadog0 @ * Caude|3 CircieciD SentryD Tasksiminny / prophet&(> Code• Issues|17 Pull requestsActions |85 Projects4 WadSecurity and quality 20 kz Insights® SettingsJy 19798 evaluation for ai activity types #468 •П Орle 9 commits into naster from JY-19798-evaluation-for-al-activity-typesP) Conversation 3B Checks 3® Fies changed 203nikolaybiaivanov commentay 5 days. ageNo description provided.G nikolaybiaivanov and others added 9 commits last week• @ JY-19798 | MIPC064644• @JY-19798 | ground truth€ 3Y-19798 | ingprove naningJY-19798 | NTP8 JY-19798 | MIPJY-19798 | MIP@ JY-19798 | ASd grouth truthB JY-39798 | Use replaysable records, fix greund trutha @Merge branch "euster' into JY-19798-evaluation-fer-ai-ectivaty-typesa936065|386294SosbeaeX M627386X scatetgnikolaybialvanov commented 5 days.222• Actions sec:( Dependabot-O Far DependolInsights & Coachin.• Dev0ux.Q Type L] to searchCode -+26,671 90000D Al BookmarksStefka StoyanovaNikolay YankovKeyn werLoading sugpestions.Stll in progress? Convert to draftAssigneesno cheestwes youtesLaoehNone yetProjectsNone yetMir stoneho mienonDevelopmentSuccess/uly merano tns pul feouest may closel5. UnsubscribeYou're receiving notifications because you'rewalchine this reposatorySteliyan GeorgievNikolay Ivanov9:48 AM Daily - PlatformW3:20...
|
NULL
|
NULL
|
NULL
|
8488
|
|
JY-20543 add AJ reports User pilot tracking by Lak JY-20543 add AJ reports User pilot tracking by LakyLak · Pull Request #11932 · jiminny/app
Platform Sprint 1 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 1 Q2 - Platform Team - Scrum Board - Jira
Jy 19798 evaluation for ai activity types by nikolaybiaivanov · Pull Request #468 · jiminny/prophet
Jy 19798 evaluation for ai activity types by nikolaybiaivanov · Pull Request #468 · jiminny/prophet
Close tab
New Tab
Customize sidebar
Open Google Gemini (⌃X)
Tabs from other devices
Open history (⇧⌘H)
Open bookmarks (⌘B)
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny...
|
Firefox
|
Jy 19798 evaluation for ai activity types by nikol Jy 19798 evaluation for ai activity types by nikolaybiaivanov · Pull Request #468 · jiminny/prophet — Work...
|
github.com/jiminny/prophet/pull/468/changes
|
8489
|
|
JY-20543 add AJ reports User pilot tracking by Lak JY-20543 add AJ reports User pilot tracking by LakyLak · Pull Request #11932 · jiminny/app
Platform Sprint 1 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 1 Q2 - Platform Team - Scrum Board - Jira
Jy 19798 evaluation for ai activity types by nikolaybiaivanov · Pull Request #468 · jiminny/prophet
Jy 19798 evaluation for ai activity types by nikolaybiaivanov · Pull Request #468 · jiminny/prophet
Close tab
New Tab
Customize sidebar
Open Google Gemini (⌃X)
Tabs from other devices
Open history (⇧⌘H)
Open bookmarks (⌘B)
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny...
|
Firefox
|
Jy 19798 evaluation for ai activity types by nikol Jy 19798 evaluation for ai activity types by nikolaybiaivanov · Pull Request #468 · jiminny/prophet — Work...
|
github.com/jiminny/prophet/pull/468/changes
|
8490
|
|
DialFilelEdit|ViewTabsBookmarksHistoryExtensionsWi DialFilelEdit|ViewTabsBookmarksHistoryExtensionsWindowHelpmeet.google.com / Daily - PlatformNikolay Yankov (Presenting)(ah)Daily - Platform - now8•inbox (2) - niП1 ОрHDC 3y 19798 eva(7 Service-DeskJr Foet AmesonD Cont | Claude) Actions secrE SSHEl Datadog* Claudel3 CircieciJy 19798 evaluation for ai activity types #468JY-19798 | MTPВ 3Y-19798 | VE20 JY-19798 | A9d grouth truth@ JY-19798 | Use replaytable records, tix ground truth@Merge branch "nuster" into JY-19798-evaluation-for-al-activity-typesnikolaybiaivanov commented 5 days age@claude••1Iclaude bot commented 5 days ago • edited -Claude finished @nikolaybiaivanov's task in 1m 54s— View jobPR Review2 Read CLAUDE.md and gather contextE Review PR changesProvide feedbackOverviewThis PRadds an evaluation framework for the al-activity-type endpoint, including:• New evaluator script src/eval/eval-al-activity-type-px• Minor update to src/eval/endpoints.py• 100 input + 100 ground truth JSON files (call transcripts + expected labels) under v2-fron-logs-replayable/•«gitignore updateC C Dependat• Actions sec:D Tasks2 Insights & Coachin2936e6S3062940SosbeaeX 8427396None yetNo milestoneSuccessfully merging this pul request may close)MorkyelNotifications8 UnsubscribeYou're receiving notfications because you'reWerchure thieseee etoparkaoant8 Lock conversation( Dependabot-O Far Dependa:с:|D DevDux.D Al BookmarksStefka StoyanovaSteliyan Georgievalik9:48 AM Daily - Platform3:288 • Tue 14 Apr 9:48:53Chat6Nikolay YankovNikolay Ivanov...
|
NULL
|
NULL
|
NULL
|
8491
|
|
FirefoxFileEoit..ViewHistoryBookmarksProfilesTools FirefoxFileEoit..ViewHistoryBookmarksProfilesToolsWindow Help• github.com/jiminny/prophet/pull/468/changesjiminny / prophet &Platform Sprint 1 Q2 - Platform Tea‹> Code© Issues !? Pull requests 6€ Agents• ActionsProjects D Wiki ® Security and quality 20 Insights Lộs Settings@ Jy 19798 evaluation for ai activ® On April 24 we'll start using GitHub Copilot interaction data for Al model training unless you opt out. Review this update and manage your preferences in your GitHub account settings.+ New TabJy 19798 evaluation for ai activity types #468 °I1 Opennikolaybiaivanov wants to merge 9 commits into master from JY-19798-evaluation-for-ai-activity-typesConversation 3• Commits 9ChecksFiles changed 203« All commits +Q Filter files....• E eval_output/ai_activity_type/e….sin - ground_truthĐ gt_-E4rcp0Bd4RvtuaTN2RY...D gt_7067cZ0Bd4RvtuaT1GT...Đ gt_7k6scZ0Bd4RvtuaT9GQ...• gt_807ncZOBd4RvtuaTumT...9t_8E6-cZOBd4RvtuaThmS...Đ gt_8k7mcZOBd4RvtuaTO2R...Đ gt_8U7VcZ0Bd4RvtuaTOW...D gt_904LcpOBd4RvtuaTyGT...• gt_9E7ycZ0Bd4RvtuaTDGR..D gt_9k4Ccp0Bd4RvtuaToGTi…..Đ 9t_9U4Ccp0Bd4RvtuaT|2R...Đ gt_D1ChcZ0BCSsQHiSAHz…D gt_E1C9cZOBCSSQHiSAtTb...• gt_EFCjcZOBCSsQHiSAQTa...D 9t_EIC7cZOBCSsQHiSAozY...Đ gt_EVC1cZOBCSsQHiSAKja…Đ gt_F1DscZ0BCSsQHiSAmT..Đ gt_FFDKCZOBCSsQHiSACja...• gt_FIDTcZ0BCSsQHiSA_TY…|Đ 9t_FVDKcZOBCSsQHiSAaD...Đ 9t_G1AZCpOBCSSQHiSAHz...D gt_G80fcZ0BahljYMX0tT03...Đ gt_g8SgcZOBRbelYNJH50...• gt_GFD9cZOBCSsQHiSAHz....Đ 9t_GIAGcpOBCSsQHiSAkza...Đ gt_gsSScZoBRbelYNJHvuD...Đ gt_GVD-cZOBCSsQHiSAUz...® This page has been optimized for large pull requests. Some browser features (like Find on Page or Select All) may not work as expected.Y eval_output/ai_activity_type/eval_datasets/v2-from-logs-replayable/ground_truth/gt_-E4rcp0Bd4RvtuaTN2RY.json0a -0,0 +1,4eval_output/ai_activity_type/eval_datasets/v2-from-logs-replayable/ground_truth/gt_7067cZ0Bd4RvtuaT1GTC.jsonoa -0,0 +1,4 cav eval output/ai activity type/eval datasets/v2-from-loqs-replayable/ground truth/qt 7k6scZ0Bd4RvtuaT9GQV.ison [L•a -0,0 +1,4 caV eval output/ai activity tvpe/eval datasets/v2-from-loas-replavable/around truth/at 807ncZ0Bd4RvtuaTumTs. ison fL∞ -0,0 +1,4 @• < 4o ll © | Daily - Platform • nowA100% C•8 • Tue 14 Apr 9:48:53Q Type to search• Awaiting approvalCode -Preview) -+25,671 mu000• 0 / 203 viewedSubmit review -Switch to single file mode+4 UuUUViewed2+ {+"ai_activity_type_reasoning": "Both Connectd rep (Jake Chweidan) and prospect (Andrew Thomas) participated and callduration is 33+ minutes, exceeding 5 minutes; therefore the criteria for 'Meeting Sat' are met.",+ ÷°"ai_activity_type": "Meeting Sat"+4 UUUUU ]Viewed |•"ai_activity_type_reasoning": "Vendor rep Toby Firkins (Connectd) and prospect Chris Lockyer both attended, and callduration is 12 minutes 11 seconds (>5 minutes), satisfying both criteria.","ai_activity_type": "Meeting Sat"+400000 _ Viewed |+ {"ai_activity_type_reasoning": "Both a Connectd rep (Alex Pertwee) and the prospect (Dan Hyde) attended the call, andthe call duration was 33 minutes 41 seconds, which is longer than 5 minutes\u2014meeting the criteria.","ai_activity_type": "Meeting Sat"+ 3+4 .0000_ Viewed+ {"ai_activity_type_reasoning": "Call metadata shows an SDR (Mads Hartmann Rasmussen) from Velatir calling Entech MarineA/S for 1m23s, typical of a short outbound cold call to book a demo. No transcript content indicating a demo, follow-up,or onboarding. The description matches an outbound cold call to book a demo.","ai_activity_type": "'Outbound Phone Call"+ }...
|
NULL
|
NULL
|
NULL
|
8492
|
|
DiaFilelEdit|ViewTabsBookmarksHistoryExtensionsWin DiaFilelEdit|ViewTabsBookmarksHistoryExtensionsWindowHelpDaily - Platform - nowmeet.google.com / Daily - PlatformNikolay Yankov (Presenting)8•Tue 14 Apr 9:48inbox (2) - niC 3y 19798 eva(7 Service-Deskje Foet Awesoe x• Cost | Claude x© Actions secre xC Ct: Dependat xActions secre X| Dependabot. xO Far Dependa:П ОpE SSHElDatadog # Claude 3 Circieci El Sentry>X Tasks Insights & Coachin= DevJy 19798 evaluation for ai activity types #468cikolaybiairanoe wants to merge 9 commits into master from JY-19798-evaluation-for-al-activity-types ?At least 1 approving review is required by reviewers with write access.Some checks were not successful1 failing, 2 successful checksncheekvO Linting g. Tres Checking / Run Elakeß & Myoy. (ouilL_ reovest) Faling eiter 57%Z successtul checks@ Buld / SonarCloud (pull request) Suscesstul in 7mv 8 SonarCioud Code Analysis Succensful in 1m - Quality Gate panvedRequiredMerging is blockedAl least 1 approving review is required by reviewers with write acoess.squasn and meroelYou can also merge this with the command line. Vite commaynd line lestrusticosStill in progress? Convert to dratt%ecommtnWritePreviewH BIE00Add your comment here…D Al BookmarksSteika StoyanovalSteliyan GeorgievwwwrwcownhthooE5 Paste, drop, or click to asd fies11 Close pull requestRemember, contributions to this repository should follow our GitHub Community OuidelinesP ProTip! Add patch or .diff to the end of URLs for Git's plaintext views.valik9:48 AM Daily - PlatformW3:328 • Tue 14 Apr 9:48:56Chat6Nikolay YankovNikolay Ivanov...
|
NULL
|
NULL
|
NULL
|
8493
|
|
JY-20543 add AJ reports User pilot tracking by Lak JY-20543 add AJ reports User pilot tracking by LakyLak · Pull Request #11932 · jiminny/app
Platform Sprint 1 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 1 Q2 - Platform Team - Scrum Board - Jira
Jy 19798 evaluation for ai activity types by nikolaybiaivanov · Pull Request #468 · jiminny/prophet
Jy 19798 evaluation for ai activity types by nikolaybiaivanov · Pull Request #468 · jiminny/prophet
Close tab
New Tab
Customize sidebar
Open Google Gemini (⌃X)
Tabs from other devices
Open history (⇧⌘H)
Open bookmarks (⌘B)
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
prophet
prophet
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
Issues
Pull requests
Repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Issues
Issues
Pull requests (6)
Pull requests
(
6
)
Agents
Agents
Actions
Actions
Projects
Projects
Wiki
Wiki
Security and quality (20)
Security and quality
(
20
)
Insights
Insights
Settings
Settings
Important update
Important update
On April 24 we'll start using GitHub Copilot interaction data for AI model training unless you opt out.
Review this update
Review this update
and manage your preferences in your
GitHub account settings
GitHub account settings
.
Dismiss banner
Jy 19798 evaluation for ai activity types #468 Edit title
Jy 19798 evaluation for ai activity types
#
468
Edit title
Preview
Preview
Awaiting approval
Awaiting approval
Code
Code
Open
nikolaybiaivanov
nikolaybiaivanov
wants to merge 9 commits into
master
master
from
JY-19798-evaluation-for-ai-activity-types
JY-19798-evaluation-for-ai-activity-types
Copy head branch name to clipboard
Lines changed: 25671 additions & 0 deletions
Conversation (3)
Conversation
(
3
)
Commits (9)
Commits
(
9...
|
Firefox
|
Jy 19798 evaluation for ai activity types by nikol Jy 19798 evaluation for ai activity types by nikolaybiaivanov · Pull Request #468 · jiminny/prophet — Work...
|
github.com/jiminny/prophet/pull/468/changes#diff-c github.com/jiminny/prophet/pull/468/changes#diff-cac29662cfcc94ab0d41bf55628c52944063d9b3d1c18f441442c3c0da7f4fde...
|
8494
|
|
JY-20543 add AJ reports User pilot tracking by Lak JY-20543 add AJ reports User pilot tracking by LakyLak · Pull Request #11932 · jiminny/app
Platform Sprint 1 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 1 Q2 - Platform Team - Scrum Board - Jira
Jy 19798 evaluation for ai activity types by nikolaybiaivanov · Pull Request #468 · jiminny/prophet
Jy 19798 evaluation for ai activity types by nikolaybiaivanov · Pull Request #468 · jiminny/prophet
Close tab
New Tab
Customize sidebar
Open Google Gemini (⌃X)
Tabs from other devices
Open history (⇧⌘H)
Open bookmarks (⌘B)
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
prophet
prophet
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
Issues
Pull requests
Repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Issues
Issues
Pull requests (6)
Pull requests
(
6
)
Agents
Agents
Actions
Actions
Projects
Projects
Wiki
Wiki
Security and quality (20)
Security and quality
(
20
)
Insights
Insights
Settings
Settings
Important update
Important update
On April 24 we'll start using GitHub Copilot interaction data for AI model training unless you opt out.
Review this update
Review this update
and manage your preferences in your
GitHub account settings
GitHub account settings
.
Dismiss banner
Jy 19798 evaluation for ai activity types #468 Edit title
Jy 19798 evaluation for ai activity types
#
468
Edit title
Preview
Preview
Awaiting approval
Awaiting approval
Code
Code
Open
nikolaybiaivanov
nikolaybiaivanov
wants to merge 9 commits into
master
master
from
JY-19798-evaluation-for-ai-activity-types
JY-19798-evaluation-for-ai-activity-types
Copy head branch name to clipboard
Lines changed: 25671 additions & 0 deletions
Conversation (3)
Conversation
(
3
)
Commits (9)
Commits
(
9
)
Checks (3)
Checks
(
3
)
Files changed (203)
Files changed
(
203
)
Pull Request Toolbar
Pull Request Toolbar
Collapse file tree
Open
Jy 19798 evaluation for ai activity types
Jy 19798 evaluation for ai activity types
#
468
All commits
All commits
nikolaybiaivanov
nikolaybiaivanov
wants to merge 9 commits into
master
master
from
JY-19798-evaluation-for-ai-activity-types
JY-19798-evaluation-for-ai-activity-types
Copy head branch name to clipboard
0
/
203
viewed
Awaiting approval
Awaiting approval
Submit review
Submit
review
Open diff view settings
Open overview panel
Open comments panel
(
0
)
Filter files…
Filter options
File tree
File tree
eval_output/ai_activity_type/eval_datasets/v2-from-logs-replayable
ground_truth
input
-E4rcp0Bd4RvtuaTN2RY.json
-E4rcp0Bd4RvtuaTN2RY.json
7067cZ0Bd4RvtuaT1GTC.json
7067cZ0Bd4RvtuaT1GTC.json
7k6scZ0Bd4RvtuaT9GQV.json
7k6scZ0Bd4RvtuaT9GQV.json
807ncZ0Bd4RvtuaTumTs.json
807ncZ0Bd4RvtuaTumTs.json
8E6-cZ0Bd4RvtuaThmSx.json...
|
Firefox
|
Jy 19798 evaluation for ai activity types by nikol Jy 19798 evaluation for ai activity types by nikolaybiaivanov · Pull Request #468 · jiminny/prophet — Work...
|
github.com/jiminny/prophet/pull/468/changes#diff-c github.com/jiminny/prophet/pull/468/changes#diff-cac29662cfcc94ab0d41bf55628c52944063d9b3d1c18f441442c3c0da7f4fde...
|
8495
|
|
DialFilelEdit|ViewTabsBookmarksHistoryExtensionsWi DialFilelEdit|ViewTabsBookmarksHistoryExtensionsWindowHelpmeet.google.com / Daily - PlatformNikolay Yankov (Presenting)> 0.(ah)Daily - Platform - now8•nbox (2) - nilC 3y 19798 eva(7 Service-DeskJr Foet AmesorCost | Claude@ Actions secrC C DependatDatados• * Claudel3 CircieciD Sentryjiminny prophet&(> Code• Issues|17 Pull requestsActions|65 Projects4 WidSecurity and quality 20 ke Insights® SettingsJy 19798 evaluation for ai activity types #468 •П Орpe 9 commits into naster from JY-19798-evaluation-for-ai-activity-typesP) Conversation 3B Checks 3® Fies changed 203nikolaybiaivanov commented 5 days agoNo description providedG nikolaybiaivanow and others added 9 commits last week• @ JY-19798 | MIPC064644• @ JY-19798 | ground truth@ 3Y-19798 | Ingprove naningJY-19798 | VTPB JY-19798 | MIP0 JY-19798 | MIP@ JY-19798 | A6d grouth truthB JY-19798 | Use replaysable records, tix ground trutha @Kerge Dranch "euster' into JY-19798-evaluation-fer-al-ectivity-typesSoabeaeX 9627386X Scatetenikolaybiaivanov commented 5 days. 292• Actions sec:( Dependabot-O Far DependolD TasksInsights & Coachin.• DevDux.Q Type L] to search• Awalting approvalCode -+25,671 80000ReviewersSuggestionsмебув-аAt least 1 approving review is required to mergethis pull request.Stll in progress? Convert to dratNo one-assign yourseltLeoehNone yetProjectsMorkTeDevelopmentSuccessluly merane ts pul reouest may closelSterka StoyanovaSteliyan Georgievalik9:49 AM Daily - PlatformW3:388 • Tue 14 Apr 9:49:02Chat6Nikolay YankovNikolay Ivanov...
|
NULL
|
NULL
|
NULL
|
8496
|
|
DialFilelEditViewTabsBookmarksHistoryExtensionsWin DialFilelEditViewTabsBookmarksHistoryExtensionsWindowHelpmeet.google.com / Daily - PlatformNikolay Yankov (Presenting)• 0.Inbox (2) - niC Jy 19798 eva(7 Service-Desk xJr Foet AmesonCost | Claude@ Actions secreC C Dependat• Actions sec:https://jiminny.atlassian.net/jira/software/c/projects/JY/boards/37?selectedissue=Jy-197980 ProjectsE SSHDatados0 * Claudel9 CircieclSentry8 TasksInsights & Coachin.Platform Team800$ JY-18679 / E JY-19798READY FOR DEV 1Evaluation for AI Activity TypeshoosnexWiwexceed Fontawesomepackage limitsMAINTENANCISReady for Dev8-20564 1 П ****=DescriptionA few months ago we launch a new feature which uses AI to detect the activity type of a call. This is done by adding a |description to each activity type in the Playbook page. We still don't have automatic evaluation for this feature.• create automaue evaluacon lor as Acuvity lype reature• use data from Jiminny and other customers as a data set• make sure we test with different teams - sales/cs and with different call types (conference, dialer calls)|• use a smart modei to determine whether the data set can be set as a source of truth |• make sure the evaluation is accurate and the results are good and reflableSUOKaSKSAdd subtaskLinked work itemsrelates to• JY-20510 Combine Al requests for summaries, action items, key points, act..ActivityAtdcotnen@3Code Review v3x Improve Technical Story~ Details• Nikolay IvanovAssign to meKODOh2 Galya DimitrovaUowoe ooneQ Open with VS Code1 branch9 commits1 pull requestToulld TatlingComponentsPlatformSub-ProductAoOrconLabels( Dependabot-• DelDux.5 days agc9:49 AM Daily - PlatformDaily - Platform - now8•O Far DependolD Al BookmarksGroup: Queriesleature fiag reieaseDeployed0 -20505 15 * ****=AI Review - Q1CTNIHEUANTANOCOEDeployed-19082 |104000-No sucn co umh "lype onentity 'Event"Deployed2038405 +000=Missed deletion of stale0 maon48009(Pipedrive) DI > status notlabeled, date data missingREDUCE CHURNDeployed0 -20614 05 П ****=:Stetka StoyanovaSteliyan Georgievvalik3:398 • Tue 14 Apr 9:49:03Chat6Nikolay YankovNikolay Ivanov...
|
NULL
|
NULL
|
NULL
|
8497
|
|
FirefoxFileEoit..ViewHistoryBookmarksProfilesTools FirefoxFileEoit..ViewHistoryBookmarksProfilesToolsWindow Help© github.com/jiminny/prophet/pull/468/changes#diff-cac29662cfcc94ab0d41bf55628c52944063d9b3d1c18f441442c3c0da7f4fdejiminny / prophet &Platform Sprint 1 Q2 - Platform Tea‹> Code• Issues 12 Pull requests 6& Agents• ActionsProjects D Wiki ® Security and quality 20 Insights Lộs Settings@ Jy 19798 evaluation for ai activ® On April 24 we'll start using GitHub Copilot interaction data for Al model training unless you opt out. Review this update and manage your preferences in your GitHub account settings.+ New TabJy 19798 evaluation for ai activity types #468 °I1 Opennikolaybiaivanov wants to merge 9 commits into master from JY-19798-evaluation-for-ai-activity-typesConversation 3• Commits 9ChecksFiles changed 203« All commits +Q Filter files....• E eval_output/ai_activity_type/e….• ground_truthinputd -E4rcpOBd4RvtuaTN2RY.jsonĐ 7067cZ0Bd4RvtuaT1GTC.js...• 7k6scZ0Bd4RvtuaT9GQVjs….Đ 807ncZ0Bd4RvtuaTumTs.js...Đ 8E6-cZOBd4RvtuaThmSx.js…..D 8k7mcZ0Bd4RvtuaTO2Rf.js...Đ 8U7VcZOBd4RvtuaTOWRt.)...• 904Lcp0Bd4RvtuaTyGTO.js...9E7ycZ0Bd4RvtuaTDGRS.js…..Đ 9k4Ccp0Bd4RvtuaToGTi.jsonĐ 9U4Ccp0Bd4RvtuaTI2Rv.jsonĐ DiChcZOBCSsQHiSAHzby.j...Đ E1C9cZ0BCSsQH¡SAtTbc.js…Đ EFCjcZOBCSsQHiSAQTaU.js..Đ ElC7cZ0BCSsQHiSAozY2.js…Đ_EVC1cZ0BCSsQHiSAKjaJ.js…Đ F1DscZOBCSsQHiSAmTa.)...• FFDKcZOBCSsQHiSACjaL.js.Đ FIDTCZOBCSsQHiSA_TYejs….Đ FVDKcZ0BCSsQHiSAaDZ-j…G1AZсp0BCSsQHiSAHzbe.j…Đ G80fcZ0BahljYMX0tT03.json• g8SgcZoBRbelYNJH5OAi.js...Đ GFD9cZ0BCSsQHiSAHzbU..Đ GIAGсp0BCSsQHiSAkzaD.jĐ gsSScZ0BRbelYNJHvuD6Jj……® This page has been optimized for large pull requests. Some browser features (like Find on Page or Select All) may not work as expected.Y eval_output/ai_activity_type/eval_datasets/v2-from-logs-replayable/ground_truth/gt_-E4rcp0Bd4RvtuaTN2RY.json0a -0,0 +1,4eval_output/ai_activity_type/eval_datasets/v2-from-logs-replayable/ground_truth/gt_7067cZ0Bd4RvtuaT1GTC.json" 0g -0,0 +1,4 Cgv eval output/ai activity type/eval datasets/v2-from-logs-replavable/ground truth/qt 7k6scZ0Bd4RvtuaT9GOV.ison [L•a -0,0 +1,4 @V eval output/ai activity tvpe/eval datasets/v2-from-loas-replavable/around truth/at 807ncZ0Bd4RvtuaTumTs. ison fL∞ -0,0 +1,4 @2• < 4o ll © | Daily - Platform • nowA100% C•8 • Tue 14 Apr 9:49:03Q Type to search• Awaiting approvalCode -Preview) -+25,671 mu000• 0 / 203 viewedSubmit review -Switch to single file mode+4 UUUULViewed+ {+"ai_activity_type_reasoning": "Both Connectd rep (Jake Chweidan) and prospect (Andrew Thomas) participated and callduration 1s 33+ minutes, exceeding minutes; theretore the criteria tor "Meeting Sat are met.",+=°"ai_activity_type": "Meeting Sat"+4 UUUUU ]Viewed |•"ai_activity_type_reasoning": "Vendor rep Toby Firkins (Connectd) and prospect Chris Lockyer both attended, and callduration is 12 minutes 11 seconds (>5 minutes), satisfying both criteria.","ai_activity_type": "Meeting Sat"+400000 _ Viewed |+ {"ai_activity_type_reasoning": "Both a Connectd rep (Alex Pertwee) and the prospect (Dan Hyde) attended the call, andthe call duration was 33 minutes 41 seconds, which is longer than 5 minutes\u2014meeting the criteria.","ai_activity_type": "Meeting Sat"+ 3+4 .0000_ Viewed+ {"ai_activity_type_reasoning": "Call metadata shows an SDR (Mads Hartmann Rasmussen) from Velatir calling Entech MarineA/S for 1m23s, typical of a short outbound cold call to book a demo. No transcript content indicating a demo, follow-up,or onboarding. The description matches an outbound cold call to book a demo.","ai_activity_type": "Outbound Phone Call"+ }...
|
NULL
|
NULL
|
NULL
|
8498
|
|
DialFilelEditViewTabsBookmarksHistoryExtensionsWin DialFilelEditViewTabsBookmarksHistoryExtensionsWindowHelpmeet.google.com / Daily - PlatformNikolay Yankov (Presenting)Inbox (2) - niC 3y 19798 eva(l Service-Desk xJr Foet AmesonD Cont | Claude@ Actions secreC C Dependat• Actions sec:https://jiminny.atlassian.net/jira/software/c/projects/JY/boards/37?selectedissue=Jy-197980 Projects#I AWS|E SSHE Datadog0 * Claudel5 CircieClSentry8 TasksInsights & Coachin.Platform Team800$ JY-18679 / E JY-19798READY FOR DEV 1Evaluation for AI Activity Typesexceed Fontawesomepockage limitsMAINTENANOSReady for Dev8-20564 1 П ****=DescriptionA few months ago we launch a new feature which uses AI to detect the activity type of a call. This is done by adding adescription to each activity type in the Playbook page. We still don't have automatic evaluation for this feature.• create automaue evaluacon lor as Acuvity lype reature• use data from Jiminny and other customers as a data set|• make sure we test with different teams - sales/cs and with different call types (conference, dialer calls)• use a smart modei to determine whether the data set can be set as a source of truth |•eheechehoecitwewhewientrrene weteecetminc• make sure the evaluation is accurate and the results are good and rellableSUDKaSKSAdd subtaskLinked work itemsrelates to• Jy-20510 Combine Al requests vor summaries, acton kems, xey poio, oActivityAddcomnen@3Code Review v*x Improve Technical Story~ Details• Nikolay IvanovAssign to meKODoh2 Galya DimitrovaUoyoe ononeQ Open with VS Code1 branch9 commits1 pull requestoutkd VatinieComponentsSub-ProductAOOrConLabels( Dependabot-• DelDux.5 days agol9:49 AM Daily - PlatformDaily - Platform - now8•O Far DependolD Al BookmarksGroup: QueriesDEPLOY &leature fiag reieaseSUPPORT TICKETSDeployed0 -20ses 15 4 •**=AI Review - Q1CTNHEUANANORCODeployed__19082 1o 400-wo sucn coumh "lype onentty 'Event"PLATTORM STARTUTTYDeployed8 -20384 05 11 ****=Missed deletion of stale0 m40n48009(Pipedrive) DI > status notlabeled, date data missingREOUCT CHURNDeployed0 -20014 05 Пl ****=Stefka StoyanovaSteliyan GeorgievValik:• 3:40|8 • Tue 14 Apr 9:49:05Chat6Nikolay YankovNikolay Ivanov...
|
NULL
|
NULL
|
NULL
|
8499
|
|
FirefoxFileEoitViewHistory BookmarksProfilesToolsW FirefoxFileEoitViewHistory BookmarksProfilesToolsWindow Help© github.com/jiminny/prophet/pull/468/changes#diff-cac29662cfcc94ab0d41bf55628c52944063d9b3d1c18f441442c3c0da7f4fdejiminny / prophet &Platform Sprint 1 Q2 - Platform Tea‹> Code• Issues 12 Pull requests 6 Agents• Actions# Projects DD Wiki ® Security and quality 20 Insights sộs Settings@ Jy 19798 evaluation for ai activ® On April 24 we'll start using GitHub Copilot interaction data for Al model training unless you opt out. Review this update and manage your preferences in your GitHub account settings.+ New TabJy 19798 evaluation for ai activity types #468 °I1 Opennikolaybiaivanov wants to merge 9 commits into master from JY-19798-evaluation-for-ai-activity-typesConversation 3a Commits 9E ChecksFiles changed 203I_ All commits -Q Filter files…..® This page has been optimized for large pull requests. Some browser features (like Find on Page or Select All) may not work as expected.• E eval_output/ai_activity_type/eval...• cround TrutnY eval_output/ai_activity_type/eval_datasets/v2-from-logs-replayable/ground_truth/gt_-E4rcp0Bd4RvtuaTN2RY.json> inputv E src/evalE endpoints.pyD eva-ar-activity-type.py* gitignore0a -0,0 +1,4eval_output/ai_activity_type/eval_datasets/v2-from-logs-replayable/ground_truth/gt_7067cZ0Bd4RvtuaT1GTC.json∞ -0,0 +1,4 c@v eval output/ai activity type/eval datasets/v2-from-logs-replavable/ground truth/qt 7k6scZ0Bd4RvtuaT9GOV.ison [L•a -0,0 +1,4 @aV eval output/ai activity tvpe/eval datasets/v2-from-loas-replavable/around truth/at 807ncZ0Bd4RvtuaTumTs. ison fL* aa -0,0 +1,4 ca2<40 ll @ | Daily - Platform • nowA100% C•8 • Tue 14 Apr 9:49:04Q Type to search• Awaiting approvalCode -Preview) -+25,671 mu000• 0 / 203 viewedSubmit review -Switch to single file mode+4 UUUULViewed+ {+"ai_activity_type_reasoning": "Both Connectd rep (Jake Chweidan) and prospect (Andrew Thomas) participated and callduration 1s 33+ minutes, exceeding minutes; theretore the criteria tor "Meeting Sat are met.",+=°"ai_activity_type": "Meeting Sat"+4 UUUUU ]Viewed ••"ai_activity_type_reasoning": "Vendor rep Toby Firkins (Connectd) and prospect Chris Lockyer both attended, and callduration is 12 minutes 11 seconds (>5 minutes), satisfying both criteria.","ai_activity_type": "Meeting Sat"+400000 _ Viewed+ {"ai_activity_type_reasoning": "Both a Connectd rep (Alex Pertwee) and the prospect (Dan Hyde) attended the call, andthe call duration was 33 minutes 41 seconds, which is longer than 5 minutes\u2014meeting the criteria.","ai_activity_type": "Meeting Sat"+ 3+4 .0000_ Viewed+ {"ai_activity_type_reasoning": "Call metadata shows an SDR (Mads Hartmann Rasmussen) from Velatir calling Entech MarineA/S for 1m23s, typical of a short outbound cold call to book a demo. No transcript content indicating a demo, follow-up,or onboarding. The description matches an outbound cold call to book a demo.","ai_activity_type": "'Outbound Phone Call"+ }...
|
NULL
|
NULL
|
NULL
|
8500
|
|
DiaFilelEditViewTabsBookmarksHistoryExtensionsWind DiaFilelEditViewTabsBookmarksHistoryExtensionsWindowHelp$0.Daily - Platform - nowmeet.google.com / Daily - PlatformNikolay Yankov (Presenting)inbox (2) - niC 3y 19798 eva(7 Service-Deskje Foet Awesoe xCont| Claude x© Actions secreC C Dependat x• Actions secr• Dependabot: xQ 8•O Far Dependas xс *11 ОрE SSHEl Dutadog # Claude O Circieci E Sentry>X 0 Tasksinsights & CoachinJy 19798 evaluation for ai activity types #468its leto naster from JY-19798-evaluation-for-al-activity-types2. Invalid modei name: "OPENAI: gpt-5_2"src/eval/eval-al-activity-type.py line 32:CROUND_TRUTH,LUM,ENOPOINT: str • "OPENAI: gpt-5_2*spt-5_2 ls not a real OpenAl model name. This looks like a typo - was this meant to be gpt-40, gpt-4-turbo, orsimilar? This would cause silent falures il -populate-ground-truth-lte is used.Fix this →3. Dead code: ground_truth_es in endpoints.pysre/eval/endpoints.py line 216:"ground_truth_es*: entry-get("ground_truth_es"),This key is passed through but nowhere in the evaluator is ground,truth,es ever set on entries —|entry-get (*ground_truth_es*) will always return None . Either populate this field in the collector or remove the key.4. Missing newline at end of filesrc/eval/eval-al-activity-type-py is missing a trailing newline (flagged in the diff). Most linters/editors will warnaoout trisFix this →• Devux.Steika StoyanovaSteliyan GeorgievMinor Notes* Pul mn transcnpts. tne 100 mous ties contain reai cat transcrois (ur conersabons). Contfm tris is scocpooie pelyour data handling policies before merging.• COMPARE_TO_GROUNO_TRUTH = *es" naming: The value "es" is used to mean "compare against originalElasticsearch response*, but the variable name and choice value are somewhat opaque. A comment or rename (e.g."original" ) could improve readabilty.The mualuator lsc loe e pold - e. collecemfrentet/ cellte en ites geie epitne beny checkina and9:49 AM | Daily - Platform• 3:44|8 • Tue 14 Apr 9:49:08ChatNikolay YankovNikolay Ivanov...
|
NULL
|
NULL
|
NULL
|
8501
|
|
DiaFilelEdit|ViewTabsBookmarksHistoryExtensionsWin DiaFilelEdit|ViewTabsBookmarksHistoryExtensionsWindowHelpmeet.google.com / Daily - PlatformNikolay Yankov (Presenting)nbox (2) - nilП1 ОрC 3y 19798 eva(7 Service-Deskj Foet Ameson X• Cost | Claude xfActions secrCCt: Dependat xE SSHEl Dutadog @ # Claude O Circieci E Sentry>X • TasksJy 19798 evaluation for ai activity types #468its into master from JY-19798-evaluation-for-al-activity-typesSee analysis details on SonarOube Cloud)))This branch has not been deployedo ocooymeneReview requiredAt least 1 approving review is required by reviewers with write access.Some checks were not successtuli1 failing, 2 successful checks1 failing check2 successtul checks viting & Tyne Checkiog /Run.FlakeB.&.MyRy.(0ul_rsgut1s), Falling atter 57s|Q Buld / SonarCloud (pull_request) Successful in 7mv @3 SonarCloud Code Analysis Successful in 1ml- Quaity Gate passecMerging is blockedSquash and mergeYou can also merpe this with the command line. Yitw.command line instrastions.Add a commentWritePreviewIEO0RequiredStll in progress? Convert to dreft9:49 AM Daily - Platform$0.| Daily - Platform • now• Actions secrinsights & Coachin_• Dependabot: xQ 8•O Far Dependa:с *• Dev• Ux.D Al BookmarksStetka StoyanovaSteliyan GeorgievKovalik3:478 • Tue 14 Apr 9:49:11Chat6Nikolay YankovNikolay Ivanov...
|
NULL
|
NULL
|
NULL
|
8502
|
|
FirefoxFileEoitViewHistory BookmarksProfilesToolsW FirefoxFileEoitViewHistory BookmarksProfilesToolsWindow Help© github.com/jiminny/prophet/pull/468/changes#diff-d2f9fea9bc428b119ae26a66453e8eab7dd0782d4a4d93075ca5ce05903d77e8jiminny / prophet &Platform Sprint 1 Q2 - Platform Tea‹> Code• Issues 12 Pull requests 6 Agents• Actions# Projects DD Wiki ® Security and quality 20 Insights sộs Settings@ Jy 19798 evaluation for ai activ® On April 24 we'll start using GitHub Copilot interaction data for Al model training unless you opt out. Review this update and manage your preferences in your GitHub account settings.+ New TabJy 19798 evaluation for ai activity types #468 °I1 Opennikolaybiaivanov wants to merge 9 commits into master from JY-19798-evaluation-for-ai-activity-typesConversation 3a Commits 9El ChecksFiles changed 203Il_ All commitsQ Filter files....® This page has been optimized for large pull requests. Some browser features (like Find on Page or Select All) may not work as expected.• E eval_output/ai_activity_type/eval...• cround TrutnY eval_output/ai_activity_type/eval_datasets/v2-from-logs-replayable/ground_truth/gt_-E4rcp0Bd4RvtuaTN2RY.json> D inputv E src/evalendpoints.py• eval-ai-activity-type.py* gitignore0a -0,0 +1,4eval_output/ai_activity_type/eval_datasets/v2-from-logs-replayable/ground_truth/gt_7067cZ0Bd4RvtuaT1GTC.json∞ -0,0 +1,4 c@v eval output/ai activity type/eval datasets/v2-from-loqs-replavable/ground truth/qt 7k6scZ0Bd4RvtuaT9GOV. ison [L•a -0,0 +1,4 @V eval output/ai activity tvpe/eval datasets/v2-from-loas-replavable/around truth/at 807ncZ0Bd4RvtuaTumTs. ison fL* aa -0,0 +1,4 ca-< 40 ll © | Daily - Platform • nowA100% C•8 • Tue 14 Apr 9:49:12Q Type to search• Awaiting approvalCode -Preview) -+25,671 m0000• 0 / 203 viewedSubmit review -Switch to single file mode+4 UUUULViewed+ {"ai_activity_type_reasoning": "Both Connectd rep (Jake Chweidan) and prospect (Andrew Thomas) participated and callduration 1s 33+ minutes, exceeding 5 minutes; theretore the criteria for "Meeting Sat are met.","ai_activity_type": "Meeting Sat"+4 UUUUU|Viewed ••"ai_activity_type_reasoning": "Vendor rep Toby Firkins (Connectd) and prospect Chris Lockyer both attended, and callduration is 12 minutes 11 seconds (>5 minutes), satisfying both criteria.","ai_activity_type": "Meeting Sat"+4 _ Viewed+ {"ai_activity_type_reasoning": "Both a Connectd rep (Alex Pertwee) and the prospect (Dan Hyde) attended the call, andthe call duration was 33 minutes 41 seconds, which is longer than 5 minutes\u2014meeting the criteria.","ai_activity_type": "Meeting Sat"+ 3+4 00000• Viewed|+ 1"ai_activity_type_reasoning": "Call metadata shows an SDR (Mads Hartmann Rasmussen) from Velatir calling Entech MarineA/S for 1m23s, typical of a short outbound cold call to book a demo. No transcript content indicating a demo, tou low-up,or onboarding. The description matches an outbound cold call to book a demo.","ai_activity_type": "Outbound Phone Call"+ }...
|
NULL
|
NULL
|
NULL
|
8503
|
|
DiaFilelEditViewTabsBookmarksHistoryExtensionsWind DiaFilelEditViewTabsBookmarksHistoryExtensionsWindowHelp$0.| Daily - Platform • nowmeet.google.com / Daily - PlatformNikolay Yankov (Presenting)Q 8•Inbox (2) - niC 3y 19798 eval(7 Service-Deskjr Foet Ameson• Cost | Claude x© Actions secrC C DependatActions sec• Dependabot: xHa Dependathttps://github.comfjminny/prophet/actions/runs/24195639151/ob/70624720594?pr=468# AWS|E3 SSHс *El Datadog % Claude5 CircieclD Sentry8 TaskslInsights & Coachin.• Dev• UxD Al BookmarksE:1@ Summary|Run Flake8 & MypyRun details© UsageWorkfiow fileRun Flake8 & Mypy•, Search logs ISteika StoyanovaHDSet up Python> • Install Poetry |> • Load cached Poetry dependencies |> © Install dependencies using Poetry|v © Run Flake8./srC/eval/eval-ai-activity-type.py:330:63: N292 no nevline at end of 11keasyncio.runtrun,cvaluatzon(alactavitytypetvaluator, args)N292 no nevline at end of fileError: Process conpleted with exit code 1.• Run Mypy |• Post Load cached Poetry dependencies |• Post Set up Python |> © Post Checkout code |> • Complete job|25sSteliyan Georgiev• 2026 0i4u6, Inc.HertrorPrivacy Security Status Communitywonest wenesouenDo not share my personal informationKovalik9:49 AM Daily - Platform3:508 • Tue 14 Apr 9:49:15Chat6Nikolay YankovNikolay Ivanov...
|
NULL
|
NULL
|
NULL
|
8504
|
|
FirefoxFileEoitViewHistory Bookmarks ProfilesTools FirefoxFileEoitViewHistory Bookmarks ProfilesToolsWindow Help© github.com/jiminny/prophet/pull/468/changes#diff-d2f9fea9bc428b119ae26a66453e8eab7dd0782d4a4d93075ca5ce05903d77e8jiminny / prophet &Platform Sprint 1 Q2 - Platform Tea‹> Code• Issues 12 Pull requests 6 Agents• Actions# Projects DD Wiki ® Security and quality 20 Insights ộ Settings@ Jy 19798 evaluation for ai activ® On April 24 we'll start using GitHub Copilot interaction data for Al model training unless you opt out. Review this update and manage your preferences in your GitHub account settings.+ New TabJy 19798 evaluation for ai activity types #468 °I1 Opennikolaybiaivanov wants to merge 9 commits into master from JY-19798-evaluation-for-ai-activity-typesConversation 3a Commits 9El ChecksFiles changed 203I_ All commits -Q Filter files....® This page has been optimized for large pull requests. Some browser features (like Find on Page or Select All) may not work as expected.• E eval_output/ai_activity_type/eval...• cround TrutnY eval_output/ai_activity_type/eval_datasets/v2-from-logs-replayable/ground_truth/gt_-E4rcp0Bd4RvtuaTN2RY.json> D input~ E src/evalendpoints.pyE eval-ai-activity-type.pyE gitignore0a -0,0 +1,4eval_output/ai_activity_type/eval_datasets/v2-from-logs-replayable/ground_truth/gt_7067cZ0Bd4RvtuaT1GTC.json∞ -0,0 +1,4 c@v eval output/ai activity type/eval datasets/v2-from-logs-replavable/ground truth/qt 7k6scZ0Bd4RvtuaT9GOV.ison [L•a -0,0 +1,4 caV eval output/ai activity tvpe/eval datasets/v2-from-loas-replavable/around truth/at 807ncZ0Bd4RvtuaTumTs. ison fL∞ -0,0 +1,4 @github.com/iiminny/prophet/pull/468/changes#diff-fc8a653dec2cd8e3d632fa52e0abd4b22176ce7619cc5752c4efe9e53f083bH N• < 4o ll © | Daily - Platform • nowA100% C8 • Tue 14 Apr 9:49:15Q Type to search• Awaiting approvalCode -Preview) -+25,671 mu000• 0 / 203 viewedSubmit review -Switch to single file mode+4 UuUUViewed+ {+"ai_activity_type_reasoning": "Both Connectd rep (Jake Chweidan) and prospect (Andrew Thomas) participated and callduration 1s 33+ minutes, exceeding minutes; theretore the criteria tor "Meeting Sat are met.","ai_activity_type": "Meeting Sat"+4 UUUUU ]Viewed ••"ai_activity_type_reasoning": "Vendor rep Toby Firkins (Connectd) and prospect Chris Lockyer both attended, and callduration is 12 minutes 11 seconds (>5 minutes), satisfying both criteria.","ai_activity_type": "Meeting Sat"+400000 _ Viewed |+ {"ai_activity_type_reasoning": "Both a Connectd rep (Alex Pertwee) and the prospect (Dan Hyde) attended the call, andthe call duration was 33 minutes 41 seconds, which is longer than 5 minutes\u2014meeting the criteria.","ai_activity_type": "Meeting Sat"+ 3+4 .0000 Viewed+ {"ai_activity_type_reasoning": "Call metadata shows an SDR (Mads Hartmann Rasmussen) from Velatir calling Entech MarineA/S for 1m23s, typical of a short outbound cold call to book a demo. No transcript content indicating a demo, follow-up,or onboarding. The description matches an outbound cold call to book a demo.","ai_activity_type": "Outbound Phone Call"+ }...
|
NULL
|
NULL
|
NULL
|
8505
|
|
DiaFilelEditViewTabsBookmarksHistoryExtensionsWind DiaFilelEditViewTabsBookmarksHistoryExtensionsWindowHelp$0.Daily - Platform - nowmeet.google.com / Daily - PlatformNikolay Yankov (Presenting)Inbox (2) - niC 3y 19798 eval(7 Service-Deskjr Foet Ameson• Cost | Claude x© Actions secrC C Dependathttps://github.comfjminny/prophet/actions/runs/24195639151/ob/70624720594?pr=468B AWSE3 SSHEl Datados * Clude5 CircieclD Sentry8 TaskslRun Flake8 & MypyActions secQ 8•• Dependabot: xHa Dependatс *Insights & Coachin.• Dev• Ux0 Al BookmarksSummary|Run Flake8 & MypyRun details© UsageWorkfiow file•, Search logs ISteika Stoyanova• Set up Python |> • Install Poetry |> • Load cached Poetry dependencies |> © Install dependencies using Poetry|• • Run Flakes g1 •Run poetry run flakeß, -count -select-E9,F63,F7,F82,N293,N291,W292 --show-source -statistics./src/eval/eval-al-activity-type.py:330:63: k292 no newiline at end of fileasyncio.runtrun,cvaluatzon(alactavityTypetvaluator, args)Error: Process conpleted with exit code 1.• Run Mypy|• Post Load cached Poetry dependencies |• Post Set up Python|› © Post Checkout code> • Comolete icbl12$25s7sSteliyan Georgiev• 2026 0i4u6, Inc.HortroPrivacy Security Status Community Docsvanest wenesouvomsureurtaine cumeuKovalik9:49 AM Daily - Platform3:518 • Tue 14 Apr 9:49:16Chat6Nikolay YankovNikolay Ivanov...
|
NULL
|
NULL
|
NULL
|
8506
|
|
Firefox FileEoitViewHistory BookmarksProfilesPla Firefox FileEoitViewHistory BookmarksProfilesPlatform Sprint 1 Q2 - Platform Tea@ Jy 19798 evaluation for ai activ *+ New labTools Window Help© github.com/jiminny/prophet/pull/468/changes#diff-fc8aG53dec2cd8e3d632fa5d32e0abd4b22176ce7619cc5752c4efe9e53f083b11 OpenJyvoeVluar on Tor ai activty tvoes #400All commitsnikolaybiaivanov wants to merge 9 commits into master from JY-19798-evaluation-for-ai-activity-types [v eval_output/ai_activity_type/eval_datasets/v2-from-logs-replayable/ground_truth/gt_7k6scZ0Bd4RvtuaT9GQV.json LQ Filter files...v eval_output/ai_activity_type/eval_datasets/v2-from-logs-replayable/ground_truth/gt_807ncZ0Bd4RvtuaTumTs.json L-• E eval_output/ai_activity_type/eval.…> • ground_truth> D input• E src/evalendpoints.py* eval-ai-activity-type.py*.gitignore0c -0,0 +1,4 CgY eval_output/ai_activity_type/eval_datasets/v2-from-logs-replayable/ground_truth/gt_8E6-cZ0Bd4RvtuaThmSx.json0a -0,0 +1,4eval_output/ai_activity_type/eval_datasets/v2-from-logs-replayable/ground_truth/gt_8k7mcZ0Bd4RvtuaT02Rf.json0c -0,0 +1,4 Cgeval_output/ai_activity_type/eval_datasets/v2-from-logs-replayable/ground_truth/gt_8U7VcZ0Bd4RvtuaT0WRt.json L0a -0,0 +1,4v eval output/ai activity type/eval datasets/v2-from-loqs-replayable/ground truth/gt 904Lcp0Bd4RvtuaTyGTO. ison [L∞a -0,0 +1,4 ca© Daily - Platform - nowA100% C8 • Tue 14 Apr 9:49:18C0 Zus viewedi• Awallind aporovaliVomitrevew+4 UUUUI_VIewea• Viewed"ai activity type reasoning": "Call metadata shows an SDR (Mads Hartmann Rasmussen) from Velatir calling Entech MarineA/S for 1m23s, typical of a short outbound cold call to book a demo. No transcript content indicating a demo, follow-up,or onboarding. The description matches an outbound cold call to book a demo.","ai_activity_type": "Outbound Phone Call"+4 UUUU!• Viewed+ 1"ai activity type reasoning": "Call metadata: 57-second call, vendor sales director Jake Bishop outbound to a contactat Hanna Instruments Ltd (a manufacturing company). No transcript content. Short scheduled call likely an initialoutreach/cold qualification attempt rather than discovery, demo, or scoping. Fits the Manufacturer cold prospectingdescription focused on quick relevance validation and booking a follow-up.","ai_activity_type": "Outreach and Qualification - Manufacturers"+ }+4-= Viewed•*•"ai_activity_type_reasoning": "Both Connectd rep (Jake Chweidan) and the prospect (Ben 0\u2019Neill) are listed asparticipants, and the call duration is 24 minutes 58 seconds, which is longerIthan 5 minutes. Therefore the criteria forthis activity type are met.","ai_activity_type": "Meeting Sat"+4 .0000 _ Viewed ••"ai_activity_type_reasoning": "Both a Connectd representative (Alex Rose) and a client participant (Kayleigh Laverick)auuended the care and the cal auramon was 25 mnures 40 seconas, whch exceeas o mnures uzort meerno boun reounreccriteria.","ai_activity_type": "Meeting Sat"+4 --- Viewed+ {"ai_activity_type_reasoning": "Call had both Connectd rep (Harry Field) and prospect (Roselle Allsop) present, but thecall duration was 4 minutes 48 seconds, which is less than the required >5 minutes threshold for the \"Meeting Sat)"activity type. Therefore the criteria are not fully met and no activity type applies.","ai_activity_type": null...
|
NULL
|
NULL
|
NULL
|
8507
|
|
DiaFilelEditViewTabsBookmarksHistoryExtensionsWind DiaFilelEditViewTabsBookmarksHistoryExtensionsWindowHelp$0.Daily - Platform - nowmeet.google.com / Daily - PlatformNikolay Yankov (Presenting)Inbox (2) - niIEB)Q 8•C 3y 19798 eval(7 Service-Deskjr Foet Ameson• Cost | Claude x© Actions secrC C DependatActions sec• Dependabot: xHa Dependatinny/prophet/actions/runs/24195639151/ob/70624720594?pr=468B AWSE3 SSHс *El Datados * Clude5 CircieclD Sentry8 TaskslInsights & Coachin.• Del• UxRun Flake8 & Mypy•, Search logs I0 Al BookmarksSummary|Run Flake8 & MypyRun details© UsageWorkfiow fileSterka StoyanovaSet up Python> • Install Poetry |> • Load cached Poetry dependencies |> © Install dependencies using Poetry|v @ Run Flake81 •Run poetry run flakeß, -count -select-E9,F63,F7,F82,N293,N291,W292 --show-source -statistics./src/eval/eval-al-activity-type.py:330:63: N292 no newline at end of fileasyncio.runtrun_cvaluatzon(AlactavityTypetvaluator, args)Error: Process conpleted with exit code 1.• Run Mypy|• Post Load cached Poetry dependencies |• Post Set up Python |› © Post Checkout code> O Complete job• 2026 0i4u6, Inc.HortroPrivacy Security Status Community Docsvonest menesn25sSteliyan GeorgievKovalik9:49 AM Daily - Platform3:588 • Tue 14 Apr 9:49:22Chat6Nikolay YankovNikolay Ivanov...
|
NULL
|
NULL
|
NULL
|
8508
|
|
Firefox FileFoit•.••+YiewHistory Bookmarks Profi Firefox FileFoit•.••+YiewHistory Bookmarks Profiles Tools Window HelpPlatform Sprint 1 Q2 - Platform Tea@ Jy 19798 evaluation for ai activ *+ New lab© github.com/jiminny/prophet/pull/468/changes#diff-fc8a653dec2cd8e3d632fa5d32e0abd4b22176ce7619cc5752c4efe9e53f083b11 OpenJyWvoeValation Torai activity Tvoes +400All commitsnikolaybiaivanov wants to merge 9 commits into master from JY-19798-evaluation-for-ai-activity-types [eval_output/ai_activity_type/eval_datasets/v2-from-logs-replayable/ground_truth/gt_FLDTcZ0BCSsQHiSA_TYe.jsonQ Filter files...0d -0,0 +1,4 CФ• E eval_output/ai_activity_type/eval….> • ground_truth> D input• E src/evalendpoints.py• eval-ai-activity-type.pyE.gitignoreY eval_output/ai_activity_type/eval_datasets/v2-from-logs-replayable/ground_truth/gt_FVDKcZ0BCSsQHiSAaDZ-•jsondd-0,0 +1,4 adv eval output/ai activity tvpe/eval datasets/v2-from-loas-replavable/around truth/at G1AZcp0BCSs0HiSAHzbe, ison rQ* aa -0,0 +1,4 caeval_output/ai_activity_type/eval_datasets/v2-from-logs-replayable/ground_truth/gt_G80fcZ0Bah[jYMX®tT03.json [..••a -0,0 +1,4 caV eval output/ai activity tvpe/eval datasets/v2-from-loas-replavable/around truth/at q8SacZ0BRbeIYNJH50Ai,ison (L* aa -0,0 +1,4 cav eval output/ai activity type/eval datasets/v2-from-loqs-replavyable/ground truth/qt GFD9cZ0BCSs0HiSAHzbU. ison [M•a -0,0 +1,4 @github.com/iiminny/prophet/pull/468/changes#diff-fc8a653dec2cd8e3d632fa5d32e0abd4b22176ce7619cc5752c4efe9e53f083bC<40l• Daily - Platform • nowA100% C8 • Tue 14 Apr 9:49:220 Zus viewed• Awallind aporovaliSubmit review -+4 UUUU|• Viewed+ {"ai activity tvoe reasoning": "The call includes one Connectd rep (Tom Zini) and a client participant (SamarieetSingh). Call duration is 18 minutes 13 seconds, which is longer than 5 minutes. Both required criteria for this activitytype are met.","ai_activity_type": "Meeting Sat"+4 UUUUViewed+ {"ai activity type reasoning": "A Connectd representative (Georqe Kimpton-Moss) and prospect(s) (Alex Elnaugh / PrinaleMedia) both attended the call, and the call duration was 32 minutes 40 seconds, which is longer than 5 minutes. Bothcriteria in the activity type definition are satisfied.","ai_activity_type": "Meeting Sat"+4 .0000 _ ViewedLJ ••<+ {"ai_activity_type_reasoning": "The call had both a Connectd rep (Barney Hall) and the prospect (James Bentley) listedas participants, and the call duration was 7 mins 23 seconds, which is longer than 5 minutes\u2014matching bothcriteria.","ai_activity_type": "Meeting Sat"+4 nuuno O Viewed"ai_activity_type_reasoning": "The call included both a Connectd rep (Daniel French) and a prospect/client (SeanTaylor) and the call duration was 11 minutes 11 seconds, which is longer than 5 minutes, meeting both criteria.","ai_activity_type": "Meeting Sat"+ 3+4 .0000 _ ViewedLJ ••<+ 1"ai_activity_type_reasoning": "Transcript includes both Connectd rep James Baker and prospect Michael Nolan activelyparticipating. call duration is 1o minutes ie seconas, which is longer chan s minutes. boch criterta in the activity typedescription are satisfied.","ai activity type": "Meeting Sat"+4 00000|• Viewed...
|
NULL
|
NULL
|
NULL
|
8509
|
|
DiaFilelEditViewTabsBookmarksHistoryExtensionsWind DiaFilelEditViewTabsBookmarksHistoryExtensionsWindowHelp$0.| Daily - Platform • nowmeet.google.com / Daily - PlatformNikolay Yankov (Presenting)Inbox (2) - niIEB)Q 8•C 3y 19798 eval(7 Service-Deskjr Foet Ameson• Cost | Claude x© Actions secrC C DependatActions sec• Dependabot: xHa Dependatinny/prophet/actions/runs/24195639151/ob/70624720594?pr=468B AWSE3 SSHс *El Datadog % Claude5 CircieclD Sentry8 TaskslInsights & Coachin.• Dev• UxRun Flake8 & Mypy•, Search logs I0 Al BookmarksSummary|Run Flake8 & MypyRun details© UsageWorkfiow fileSterka StoyanovaSet up Python> • Install Poetry |> • Load cached Poetry dependencies |> © Install dependencies using Poetry|v @ Run Flake81 •Run poetry run flakeß, -count -select-E9,F63,F7,F82,N293,N291,W292 --show-source -statistics./src/eval/eval-al-activity-type.py:330:63: N292 no newline at end of fileasyncio.runtrun_cvaluatzon(AlactavityTypetvaluator, args)Error: Process conpleted with exit code 1.• Run Mypy|• Post Load cached Poetry dependencies |• Post Set up Python |› © Post Checkout code> O Complete job• 2026 0i4u6, Inc.HortroPrivacy Security Status Community Docsvonest menesn25sSteliyan GeorgievKovalik9:49 AM Daily - Platform3:598 • Tue 14 Apr 9:49:23Chat6Nikolay YankovNikolay Ivanov...
|
NULL
|
NULL
|
NULL
|
8510
|
|
Firefox FileFoitYiewHistory BookmarksProfiles To Firefox FileFoitYiewHistory BookmarksProfiles Tools Window Help•.••+Platform Sprint 1 Q2 - Platform Tea@ Jy 19798 evaluation for ai activ *+ New lab© github.com/jiminny/prophet/pull/468/changes#diff-fc8aG53dec2cd8e3d632fa5d32e0abd4b22176ce7619cc5752c4efe9e53f083b11 OpenJyvoeVluar on Tor ai activty tvoes #400All commitsnikolaybiaivanov wants to merge 9 commits into master from JY-19798-evaluation-for-ai-activity-types [eval_output/ai_activity_type/eval_datasets/v2-from-logs-replayable/ground_truth/gt_FlDTcZ0BCSsQHiSA_TYe•json [®Q Filter files...0d -0,0 +1,4 CФ• E eval_output/ai_activity_type/eval….> • ground_truth> D input• E src/evalendpoints.py* eval-ai-activity-type.pyE .gitignoreY eval_output/ai_activity_type/eval_datasets/v2-from-logs-replayable/ground_truth/gt_FVDKcZ0BCSsQHiSAaDZ-•jsondd-0,0 +1,4 adv eval output/ai activity tvpe/eval datasets/v2-from-loas-replavable/around truth/at G1AZcp0BCSs0HiSAHzbe, ison Q* aa -0,0 +1,4 caeval_output/ai_activity_type/eval_datasets/v2-from-logs-replayable/ground_truth/gt_G80fcZ0Bah[jYMX®tT03.json [•a -0,0 +1,4 caV eval output/ai activity tvpe/eval datasets/v2-from-loas-replavable/around truth/at q8SacZ0BRbeIYNJH50Ai.ison fL* aa -0,0 +1,4 cav eval output/ai activity type/eval datasets/v2-from-loqs-replayable/ground truth/qt GFD9cZ0BCSs0HiSAHzbU. ison [M•a -0,0 +1,4 @a• иC<40l• Daily - Platform • nowA100% C• Tue 14 Apr 9:49:23C0 Zus viewedi• Awallind aporovaliSubmit review -+4 UUUU• Viewed+ {"ai activity tvoe reasoning": "The call includes one Connectd rep (Tom Zini) and a client participant (SamarieetSingh). Call duration is 18 minutes 13 seconds, which is longer than 5 minutes. Both required criteria for this activitytype are met.","ai_activity_type": "Meeting Sat"+4 UUUU.Viewed+ {"ai activity type reasoning": "A Connectd representative (Georqe Kimpton-Moss) and prospect(s) (Alex Elnaugh / PrinaleMedia) both attended the call, and the call duration was 32 minutes 40 seconds, which is longer than 5 minutes. Bothcriteria in the activity type definition are satisfied.","ai_activity_type": "Meeting Sat"+400000 _ Viewed ••+ {"ai_activity_type_reasoning": "The call had both a Connectd rep (Barney Hall) and the prospect (James Bentley) listedas participants, and the call duration was 7 mins 23 seconds, which is longer than 5 minutes\u2014matching bothcriteria.","ai_activity_type": "Meeting Sat"+4 nuuno O Viewed"ai_activity_type_reasoning": "The call included both a Connectd rep (Daniel French) and a prospect/client (SeanTaylor) and the call duration was 11 minutes 11 seconds, which is longer than 5 minutes, meeting both criteria.","ai_activity_type": "Meeting Sat"+ 3+400000 _ Viewed ••+ 1"ai_activity_type_reasoning": "Transcript includes both Connectd rep James Baker and prospect Michael Nolan activelyparticipating. call duration is 1o minutes ie seconas, which is longer chan s minutes. boch criterta in the activity typedescription are satisfied.","ai activity type": "Meetina Sat"+4 00000|• Viewed...
|
NULL
|
NULL
|
NULL
|
8511
|
|
Firefox FileEoitView History Bookmarks Profiles Firefox FileEoitView History Bookmarks Profiles Tools Window Help→Platform Sprint 1 Q2 - Platform Tea@ Jy 19798 evaluation for ai activ *+ New labI1 Open© github.com/jiminny/prophet/pull/468/changes#diff-bc37d034bad564583790a46f19d807abfe519c5671395fd494d8cce506c42947Jy 19798 evaluation for ai activity types #468All commitsnikolaybiaivanov wants to merge 9 commits into master from JY-19798-evaluation-for-ai-activity-typesv src/eval/eval-ai-activity-type.pyQ Filter files...• E eval_output/ai_activity_type/eval….> • ground_truth> D input~ & src/evalendpoints.pyD eval-ai-activity-type.pyE .gitignore• < 4o ll © | Daily - Platform • nowA100% C• 8• Tue 14 Apr 9:49:24• 0 / 203 viewed• Awallind aporovaliVomitrevew+330 UUUUU• Viewed304305306307308309310312313314315316317318319321action="store_true",dest="populate_ground_truth_llm",parser. add_argument("--ground-truth-llm-endpoint",type=str,default="OPENAI: gpt-5_2",dest="ground_truth_llm_endpoint",parser.add_argument("--compare-to-ground-truth",type=str,choices=["es", "1lm"],default="es",dest="compare_to_ground_truth",+ + +++ + + +args = parser.parse_args()AiActivityTypeEvaluator.DATA_SET = args.data_setAiActivityTypeEvaluator.POPULATE GROUND_TRUTH_LLM = bool(args.populate ground_truth_llmAiActivitvTvoeEvaluator.GROUND TRUTH LLM ENDPOINT = args.around truth 1lm endpointAiActivityTypeEvaluator.COMPARE TO GROUND TRUTH = arqs.compare to ground truthasyncio.run(run evaluation(AiActivityTypeEvaluator, arqs))V .gitignore @*C -16,6 +16,10 @@ eval_output/identify_participants/*eval_output/identify_participants/eval_datasets/*leval_output/identify_participants/eval_datasets/v4-en/leval outout/1dentity particivants/eva datasets/v4-non-en/+4 UUUUU_ Viewed22historyaws-creas.envprompt.txteval_output/identify_participants/eval_datasets/*leval output/identify participants/eval datasets/v4-en/leval_output/identify_participants/eval_datasets/v4-non-en/+ eval_output/ai_activity_type/eval_datasets/*+eva ououval aculvit tiereval caraseus/vi-tron-loos/+ leval_output/ai_activity_type/eval_datasets/v1-from-ogs/**.historyaws-creas.envprompt.txtYou made it to the end!...
|
NULL
|
NULL
|
NULL
|
8512
|
|
DiaFilelEditViewTabsBookmarksHistoryExtensionsWind DiaFilelEditViewTabsBookmarksHistoryExtensionsWindowHelp$0.| Daily - Platform • nowmeet.google.com / Daily - PlatformNikolay Yankov (Presenting)Inbox (2) - niC 3y 19798 eval(7 Service-Deskjr Foet Ameson• Cost | Claude x© Actions secrC C Dependatinny/prophet/actions/runs/24195639151/ob/70624720594?pr=468# AWS|E3 SSHEl Datadog % Claude5 CircieclD Sentry8 TaskslActions secQ 8•• Dependabot: xHa Dependatс *Insights & Coachin.• Dev• Ux0 Al BookmarksIEB)Summary|Run Flake8 & MypyRun details© UsageWorkfiow fileRun Flake8 & Mypy•, Search logs ISterka StoyanovaSet up Python> • Install Poetry |> • Load cached Poetry dependencies |> © Install dependencies using Poetry|v @ Run Flake81 •Run poetry run flakeß, -count -select-E9,F63,F7,F82,N293,N291,W292 --show-source -statistics•/src/eval/eval-ai-activity-type.py:330:63: N292 no nevline at end of fileasyncio.runtrun_cvaluatzon(AlactavityTypetvaluator, args)Error: Process conpleted with exit code 1.• Run Mypy|• Post Load cached Poetry dependencies |• Post Set up Python |› © Post Checkout code> O Complete job• 2026 0i4u6, Inc.HortroPrivacy Security Status Community Docsvonest menesnDo not share my personal information)25sSteliyan GeorgievKovalik9:49 AM Daily - Platform4:038 • Tue 14 Apr 9:49:27Chat6Nikolay YankovNikolay Ivanov...
|
NULL
|
NULL
|
NULL
|
8513
|
|
Firefox FileEoitView History Bookmarks Profiles Firefox FileEoitView History Bookmarks Profiles Tools Window Help→Platform Sprint 1 Q2 - Platform Tea@ Jy 19798 evaluation for ai activ *+ New labI1 Open© github.com/jiminny/prophet/pull/468/changes#diff-bc37d034bad564583790a46f19d807abfe519c5671395fd494d8cce506c42947Jy 19798 evaluation for ai activity types #468All commitsnikolaybiaivanov wants to merge 9 commits into master from JY-19798-evaluation-for-ai-activity-types ®v src/eval/eval-ai-activity-type.pyQ Filter files...• E eval_output/ai_activity_type/eval….> • ground_truth> D input• E src/evalendpoints.pyD eval-ai-activity-type.pyE gitignore• < 4o ll © | Daily - Platform • nowA100% C• 8 • Tue 14 Apr 9:49:27• 0 / 203 viewed• Awallind aporovaliVomitrevew+330 UUUUU• Viewed304305306307308309310312313314315316317318319321322323324325326327328329330action="store_true",dest="populate_ground_truth_llm",parser.add_argument("--ground-truth-llm-endpoint",type=str,default="OPENAI: gpt-5_2",dest="ground_truth_llm_endpoint",parser.add_argument ("--compare-to-ground-truth",type=str,choices=["es", "1lm"],default="es",dest="compare_to_ground_truth",args = parser.parse_args()AiActivityTypeEvaluator.DATA_SET = args.data_setAiActivityTypeEvaluator.POPULATE GROUND_TRUTH_LLM = bool(args.populate ground_truth_llmAiActivitvTvoeEvaluator.GROUND TRUTH LLM ENDPOINT = args.around truth 1lm endpointAiActivityTypeEvaluator.COMPARE_TO_GROUND_TRUTH = args.compare_to_ground_truthasyncio.run(run evaluation(AiActivityTypeEvaluator, arqs))V .gitignore @*C -16,6 +16,10 @@ eval_output/identify_participants/*eval_output/identify_participants/eval_datasets/*leval_output/identify_participants/eval_datasets/v4-en/leval_output/identify_participants/eval_datasets/v4-non-en/*A Bnn LViewed...historyaws-creas.envprompt.txteval_output/identify_participants/eval_datasets/*leval output/identify participants/eval datasets/v4-en/leval_output/identify_participants/eval_datasets/v4-non-en/+ eval_output/ai_activity_type/eval_datasets/*+eva ououval aculvit tiereval caraseus/vi-tron-loos/+ leval_output/ai_activity_type/eval_datasets/v1-from-logs/**.historyaws-creas.envprompt.txtYou made it to the end!...
|
NULL
|
NULL
|
NULL
|
8514
|
|
DiaFilelEditViewTabsBookmarksHistoryExtensionsWind DiaFilelEditViewTabsBookmarksHistoryExtensionsWindowHelp$0.| Daily - Platform • nowmeet.google.com / Daily - PlatformNikolay Yankov (Presenting)Inbox (2) - niIEB)Q 8•C 3y 19798 eval(7 Service-Deskjr Foet Ameson• Cost | Claude x©Actions secrC C DependatActions sec• Dependabot: xHa Dependatinny/prophet/actions/runs/24195639151/ob/70624720594?pr=468# AWS|E3 SSHс *El Datadog % Claude5 CircieclD Sentry8 TaskslInsights & Coachin.• Dev• UxRun Flake8 & Mypy•, Search logs I0 Al BookmarksSummary|Run Flake8 & MypyRun details© UsageWorkfiow fileSterka StoyanovaSet up Python> • Install Poetry |> • Load cached Poetry dependencies |> © Install dependencies using Poetry|v @ Run Flake81 •Run poetry run flakeß, -count -select-E9,F63,F7,F82,N293,N291,W292 --show-source -statistics./src/eval/eval-al-activity-type.py:330:63: N292 no newline at end of fileasyncio.runtrun_cvaluatzon(AlactavityTypetvaluator, args)Error: Process conpleted with exit code 1.• Run Mypy|• Post Load cached Poetry dependencies |• Post Set up Python |› © Post Checkout code> O Complete job• 2026 0i4u6, Inc.HortroPrivacy Security Status Community Docsvonest menesn25sSteliyan GeorgievKovalik9:49 AM Daily - Platform4:048 • Tue 14 Apr 9:49:28Chat6Nikolay YankovNikolay Ivanov...
|
NULL
|
NULL
|
NULL
|
8515
|
|
Firefox File Edit View History Bookmarks Profiles Firefox File Edit View History Bookmarks Profiles Tools Window Help•••+→ cPlatform Sprint 1 Q2 - Platform Tea( Jy 19798 evaluation for ai activ XNew 1aoi? Open©github.com/jiminny/prophet/pull/468/changes#diff-bc37d034bad564583790a46f19d807abfe519c5671395fd494d8cce506c42947Jy 19798 evaluation for ai activity types #468All commitsnikolaybiaivanov wants to merge 9 commits into master from JY-19798-evaluation-for-ai-activity-types [v src/eval/eval-ai-activity-type.pyQ Filter files...• E eval_output/ai_activity_type/eval….> D ground_truthinput• E src/evalE endpoints.pyD eval-ai-activity-type.py* gitignore•< 4o ll • | Daily - Platform • now100% C• &• Tue 14 Apr 9:49:28• 1/ 203 viewed• Awallino aporovaliVomitrevew• Viewed292293295296297298300301302303304305306307308309310311312313314315316317318319320322323324326327+ if _name_ = "__main_":parser = create parser(AiActivityTypeEvaluator)parser.add_argument("-data-set",type=str,default="initial",dest="data_set",help="Dataset folder under eval output/ai activity_type/eval_datasets/ (default: initial)",parser.add arqument("—populate-ground-truth-1lm",detsone store_true,dest="populate_ground_truth_llm",narser.add araument("--ground-truth-llm-endpoint",type=str,default="OPENAI: gpt-5_2",dest="ground_truth_llm_endpoant,parser.add_argument("-compare-to-ground-truth",type=str,choices=["es", "1lm"],default="es",dest="compare_to_ground_truth",args = parser.parse_args()AiActivityTypeEvaluator.DATA_SET = args.data_setAiActivityTypeEvaluator.POPULATE_GROUND_TRUTH_LLM = bool(args.populate_ground_truth_llm)AiActivityTypeEvaluator.GROUND_TRUTH_LLM_ENDPOINT = args.ground_truth_llm_endpointAiActivityTypeEvaluator.COMPARE_TO_GROUND_TRUTH = args.compare_to_ground_truthasyncio.run(run_evaluation(AiActivityTypeEvaluator, args))> .gitignore & ++4-0== ViewedYou made it to the end!...
|
NULL
|
NULL
|
NULL
|
8516
|
|
JY-20543 add AJ reports User pilot tracking by Lak JY-20543 add AJ reports User pilot tracking by LakyLak · Pull Request #11932 · jiminny/app
Platform Sprint 1 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 1 Q2 - Platform Team - Scrum Board - Jira
Jy 19798 evaluation for ai activity types by nikolaybiaivanov · Pull Request #468 · jiminny/prophet
Jy 19798 evaluation for ai activity types by nikolaybiaivanov · Pull Request #468 · jiminny/prophet
Close tab
New Tab
Customize sidebar
Open Google Gemini (⌃X)
Tabs from other devices
Open history (⇧⌘H)
Open bookmarks (⌘B)
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
prophet
prophet
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
Issues
Pull requests
Repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Issues
Issues
Pull requests (6)
Pull requests
(
6
)
Agents
Agents
Actions
Actions
Projects
Projects
Wiki
Wiki
Security and quality (20)
Security and quality
(
20
)
Insights
Insights
Settings
Settings
Important update
Important update
On April 24 we'll start using GitHub Copilot interaction data for AI model training unless you opt out.
Review this update
Review this update
and manage your preferences in your
GitHub account settings
GitHub account settings
.
Dismiss banner
Jy 19798 evaluation for ai activity types #468 Edit title
Jy 19798 evaluation for ai activity types
#
468
Edit title
Preview
Preview
Awaiting approval
Awaiting approval
Code
Code
Open
nikolaybiaivanov
nikolaybiaivanov
wants to merge 9 commits into
master
master
from
JY-19798-evaluation-for-ai-activity-types
JY-19798-evaluation-for-ai-activity-types
Copy head branch name to clipboard
Lines changed: 25671 additions & 0 deletions
Conversation (3)
Conversation
(
3
)
Commits (9)
Commits
(
9
)
Checks (3)
Checks
(
3
)
Files changed (203)
Files changed
(
203
)
Pull Request Toolbar
Pull Request Toolbar
Collapse file tree
Open
Jy 19798 evaluation for ai activity types
Jy 19798 evaluation for ai activity types
#
468
All commits
All commits
nikolaybiaivanov
nikolaybiaivanov
wants to merge 9 commits into
master
master
from
JY-19798-evaluation-for-ai-activity-types
JY-19798-evaluation-for-ai-activity-types
Copy head branch name to clipboard
1
/
203
viewed
Awaiting approval
Awaiting approval
Submit review
Submit
review
Open diff view settings
Open overview panel
Open comments panel
(
0
)
Filter files…
Filter options
File tree
File tree
eval_output/ai_activity_type/eval_datasets/v2-from-logs-replayable
ground_truth
input
src/eval
endpoints.py
endpoints.py
eval-ai-activity-type.py
eval-ai-activity-type.py
.gitignore
.gitignore
This page has been optimized for large pull requests.
This page has been optimized for large pull requests.
This page has been optimized for large pull requests. Some browser features (like Find on Page or Select All) may not work as expected.
Switch to single file mode
Switch to single file mode
Collapse file
src/eval/endpoints.py
src/eval/endpoints.py
src/eval/endpoints.py
Copy file name to clipboard
Expand all lines: src/eval/endpoints.py
Lines changed: 5 additions & 0 deletions
Not Viewed
Viewed
Comment on this file
More options
Original file line number
Original file line
Diff line number
Diff line change
@@ -172,6 +172,7 @@ async def process_entry(idx: int, entry: Dict[str, Any]) -> None:
172
known_keys
=
{
172
known_keys
=
{
173
"content"
,
"summary"
,
"themes"
,
"action_items"
,
173
"content"
,
"summary"
,
"themes"...
|
Firefox
|
Jy 19798 evaluation for ai activity types by nikol Jy 19798 evaluation for ai activity types by nikolaybiaivanov · Pull Request #468 · jiminny/prophet — Work...
|
github.com/jiminny/prophet/pull/468/changes#diff-f github.com/jiminny/prophet/pull/468/changes#diff-fc8a653dec2cd8e3d632fa5d32e0abd4b22176ce7619cc5752c4efe9e53f083b...
|
8517
|
|
Firefox FileEoitYiewHistory BookmarksProfilesToo Firefox FileEoitYiewHistory BookmarksProfilesTools Window Help→Platform Sprint 1 Q2 - Platform Tea( Jy 19798 evaluation for ai activ X+ New labI1 Open©github.com/jiminny/prophet/pull/468/changes#diff-fc8a653dec2cd8e3d632fa5d32e0abd4b22176ce7619cc5752c4efe9e53f083bJy 19798 evaluation for ai activity types #468All commitsnikolaybiaivanov wants to merge 9 commits into master from JY-19798-evaluation-for-ai-activity-types [[PASSWORD_DOTS]Q Filter files...• E eval_output/ai_activity_type/eval….> • ground_truth> L input.• E src/evalE endpoints.py* eval-ai-activity-type.pyE .gitignorev src/eval/eval-ai-activity-type.py L@a -0,0 +1,330 @→0 • Daily - Platform • nowA100% C8 • Tue 14 Apr 9:49:34• 1/ 203 viewed• Awallind aporovali+330 -0000• Viewed|+ #!/usr/bin/env pythons+ import asyncio+ import Json+ import os+ from pathlib import Path+ from typing import Any, Dict, List+ Import alohttp+ from src.component.call_processing.schemes.ai_activity_type_request import AiActivityTypeRequest+ from src.component.call_processing.schemes.ai activity_ type_response import AiActivityTypeResponse+Tron src,eval,aostract evalvaror imbort Aostracuevaluaton+ from src.eval.collectors import collect_test_set as es_collect_test_set+ from src.eval.common_evaluator_runner import create_parser, run_evaluation+ from src.eval.endpoints import run_endpoint as run_endpoint_http+Trom src.eval.models 1mport evaluacionkesuut22 +28+ class AiActivityTypeEvaluator(AbstractEvaluator):ENDPOINT_PATH = "call/ai-activity-type"REQUEST_MODEL = AiActivityTypeRequestRESPONSE_MODEL = AiActivityTypepfsponseOUTPUT_SUBFOLDER = "ai_activity_type"USES_LLM_EVALUATION = FalseREADS_FROM_DATASET_FOLDER = TrueDATA_SET: str = "initial"MINIMUN REOUES BODY SI4E: 1nt = vSKLPLUNKEPLATABLE_UN_COLLECI: boOl = IrueREPLAYABILITY_CHECK_TIMEOUT_SECONDS: float = 20.0POPULATE_GROUND_TRUTH_LLM: bool = FalseGROUND_TRUTH_LLM_ENDPOINT: str = "OPENAI:gpt-5_2"COMPARE_TO_GROUND_TRUTH: str = "es"def _init_(self, **kwargs: Any) -> None:super().__init_(**kwargs)_repo_root = Path(_file_).parents[2]_datasets_root = _repo_root / "eval_output" / self.OUTPUT_SUBFOLDER / "eval_datasets"_dataset_dir = _datasets_root / self._class_.DATA_SETself. dataset dir = dataset dirself._inputs_dir =_dataset_dir / "input"self._ground_truths_dir =_dataset_dir / "ground_truth"async def collect test set(self) →> List[Dict[str, Anyl]:if self.es host name:203...
|
NULL
|
NULL
|
NULL
|
8518
|
|
JY-20543 add AJ reports User pilot tracking by Lak JY-20543 add AJ reports User pilot tracking by LakyLak · Pull Request #11932 · jiminny/app
Platform Sprint 1 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 1 Q2 - Platform Team - Scrum Board - Jira
Jy 19798 evaluation for ai activity types by nikolaybiaivanov · Pull Request #468 · jiminny/prophet
Jy 19798 evaluation for ai activity types by nikolaybiaivanov · Pull Request #468 · jiminny/prophet
Close tab
New Tab
Customize sidebar
Open Google Gemini (⌃X)
Tabs from other devices
Open history (⇧⌘H)
Open bookmarks (⌘B)
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
prophet
prophet
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
Issues
Pull requests
Repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Issues
Issues
Pull requests (6)
Pull requests
(
6
)
Agents
Agents
Actions
Actions
Projects
Projects
Wiki
Wiki
Security and quality (20)
Security and quality
(
20
)
Insights
Insights
Settings
Settings
Important update
Important update
On April 24 we'll start using GitHub Copilot interaction data for AI model training unless you opt out.
Review this update
Review this update
and manage your preferences in your
GitHub account settings
GitHub account settings
.
Dismiss banner
Jy 19798 evaluation for ai activity types #468 Edit title
Jy 19798 evaluation for ai activity types
#
468
Edit title
Preview
Preview
Awaiting approval
Awaiting approval
Code
Code
Open
nikolaybiaivanov
nikolaybiaivanov
wants to merge 9 commits into
master
master
from
JY-19798-evaluation-for-ai-activity-types
JY-19798-evaluation-for-ai-activity-types
Copy head branch name to clipboard
Lines changed: 25671 additions & 0 deletions
Conversation (3)
Conversation
(
3
)
Commits (9)
Commits
(
9
)
Checks (3)
Checks
(
3
)
Files changed (203)
Files changed
(
203
)
Pull Request Toolbar
Pull Request Toolbar
Collapse file tree
Open
Jy 19798 evaluation for ai activity types
Jy 19798 evaluation for ai activity types
#
468
All commits
All commits
nikolaybiaivanov
nikolaybiaivanov
wants to merge 9 commits into
master
master
from
JY-19798-evaluation-for-ai-activity-types
JY-19798-evaluation-for-ai-activity-types
Copy head branch name to clipboard
1
/
203
viewed
Awaiting approval
Awaiting approval
Submit review
Submit
review
Open diff view settings
Open overview panel
Open comments panel
(
0
)
Filter files…
Filter options
File tree
File tree
eval_output/ai_activity_type/eval_datasets/v2-from-logs-replayable
ground_truth
input
src/eval
endpoints.py
endpoints.py
eval-ai-activity-type.py
eval-ai-activity-type.py
.gitignore
.gitignore
This page has been optimized for large pull requests.
This page has been optimized for large pull requests.
This page has been optimized for large pull requests. Some browser features (like Find on Page or Select All) may not work as expected.
Switch to single file mode
Switch to single file mode
Collapse file
src/eval/endpoints.py
src/eval/endpoints.py
src/eval/endpoints.py
Copy file name to clipboard
Expand all lines: src/eval/endpoints.py
Lines changed: 5 additions & 0 deletions
Not Viewed
Viewed
Comment on this file
More options
Original file line number
Original file line
Diff line number
Diff line change
@@ -172,6 +172,7 @@ async def process_entry(idx: int, entry: Dict[str, Any]) -> None:
172
known_keys
=
{
172
known_keys
=
{
173
"content"
,
"summary"
,
"themes"
,
"action_items"
,
173
"content"
,
"summary"
,
"themes"
,
"action_items"
,
174
"salesRep"
,
"customer"
,
174
"salesRep"
,
"customer"
,
175
+
"ai_activity_type"
,
"ai_activity_type_reasoning"
,
175
}
176
}
176
is_empty
=
known_keys
.
isdisjoint
(
new_data
.
keys
())...
|
Firefox
|
Jy 19798 evaluation for ai activity types by nikol Jy 19798 evaluation for ai activity types by nikolaybiaivanov · Pull Request #468 · jiminny/prophet — Work...
|
github.com/jiminny/prophet/pull/468/changes#diff-f github.com/jiminny/prophet/pull/468/changes#diff-fc8a653dec2cd8e3d632fa5d32e0abd4b22176ce7619cc5752c4efe9e53f083b...
|
8519
|
|
DiaFilelEditViewTabsBookmarksHistoryExtensionsWind DiaFilelEditViewTabsBookmarksHistoryExtensionsWindowHelp$0.| Daily - Platform • nowmeet.google.com / Daily - PlatformNikolay Yankov (Presenting)Inbox (2) - niE:1Q 8•C 3y 19798 eval(7 Service-Deskjr Foet Ameson• Cost | Claude x©Actions secrC C DependatActions sec• Dependabot: xHa Dependatinny/prophet/actions/runs/24195639151/ob/70624720594?pr=468B AWSE3 SSHс *El Datadog % Claude5 CircieclD Sentry8 TaskslInsights & Coachin.• Dev• UxRun Flake8 & Mypy•, Search logs I0 Al BookmarksSummary|Run Flake8 & MypyRun details© UsageWorkfiow fileStefka StoyanovaHDSet up Python> • Install Poetry |> • Load cached Poetry dependencies |> © Install dependencies using Poetry|v @ Run Flake81 •Run poetry run flakeß, -count -select-E9,F63,F7,F82,N293,N291,W292 --show-source -statistics./src/eval/eval-al-activity-type.py:330:63: N292 no newline at end of fileasyncio.runtrun_cvaluatzon(AlactavityTypetvaluator, args)Error: Process conpleted with exit code 1.• Run Mypy|• Post Load cached Poetry dependencies |• Post Set up Python |› © Post Checkout code> O Complete job• 2026 0i4u6, Inc.HortroPrivacy Security Status Community Docsvonest menesn12$1s25s7sSteliyan Georgievovalik9:49 AM Daily - Platform4:158 • Tue 14 Apr 9:49:40Chat6Nikolay YankovNikolay Ivanov...
|
NULL
|
NULL
|
NULL
|
8520
|
|
JY-20543 add AJ reports User pilot tracking by Lak JY-20543 add AJ reports User pilot tracking by LakyLak · Pull Request #11932 · jiminny/app
Platform Sprint 1 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 1 Q2 - Platform Team - Scrum Board - Jira
Jy 19798 evaluation for ai activity types by nikolaybiaivanov · Pull Request #468 · jiminny/prophet
Jy 19798 evaluation for ai activity types by nikolaybiaivanov · Pull Request #468 · jiminny/prophet
Close tab
New Tab
Customize sidebar
Open Google Gemini (⌃X)
Tabs from other devices
Open history (⇧⌘H)
Open bookmarks (⌘B)
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
prophet
prophet
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
Issues
Pull requests
Repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Issues
Issues
Pull requests (6)
Pull requests
(
6
)
Agents
Agents
Actions
Actions
Projects
Projects
Wiki
Wiki
Security and quality (20)
Security and quality
(
20
)
Insights
Insights
Settings
Settings
Important update
Important update
On April 24 we'll start using GitHub Copilot interaction data for AI model training unless you opt out.
Review this update
Review this update
and manage your preferences in your
GitHub account settings
GitHub account settings
.
Dismiss banner
Jy 19798 evaluation for ai activity types #468 Edit title
Jy 19798 evaluation for ai activity types
#
468
Edit title
Preview
Preview
Awaiting approval
Awaiting approval
Code
Code
Open
nikolaybiaivanov
nikolaybiaivanov
wants to merge 9 commits into
master
master
from
JY-19798-evaluation-for-ai-activity-types
JY-19798-evaluation-for-ai-activity-types
Copy head branch name to clipboard
Lines changed: 25671 additions & 0 deletions
Conversation (3)
Conversation
(
3
)
Commits (9)
Commits
(
9
)
Checks (3)
Checks
(
3
)
Files changed (203)
Files changed
(
203
)
Pull Request Toolbar
Pull Request Toolbar
Collapse file tree
Open
Jy 19798 evaluation for ai activity types
Jy 19798 evaluation for ai activity types
#
468
All commits
All commits
nikolaybiaivanov
nikolaybiaivanov
wants to merge 9 commits into
master
master
from
JY-19798-evaluation-for-ai-activity-types
JY-19798-evaluation-for-ai-activity-types
Copy head branch name to clipboard
1
/
203
viewed
Awaiting approval
Awaiting approval
Submit review
Submit
review
Open diff view settings
Open overview panel
Open comments panel
(
0
)
Filter files…
Filter options
File tree
File tree
eval_output/ai_activity_type/eval_datasets/v2-from-logs-replayable
ground_truth
input
src/eval
endpoints.py
endpoints.py
eval-ai-activity-type.py
eval-ai-activity-type.py
.gitignore
.gitignore
This page has been optimized for large pull requests.
This page has been optimized for large pull requests.
This page has been optimized for large pull requests. Some browser features (like Find on Page or Select All) may not work as expected.
Switch to single file mode
Switch to single file mode
Collapse file
src/eval/eval-ai-activity-type.py
src/eval/eval-ai-activity-type.py
src/eval/eval-ai-activity-type.py
Copy file name to clipboard
Lines changed: 330 additions & 0 deletions
Not Viewed
Viewed
Comment on this file
More options
Original file line number
Original file line
Diff line number
Diff line change
@@ -0,0 +1,330 @@
1
+
#!/usr/bin/env python3
2
+
import
asyncio
3
+
import
json
4
+
import
os
5
+
from
pathlib
import
Path
6
+
from
typing
import
Any
,
Dict
,
List
7
+
8
+
import
aiohttp
9
+
10
+
from
src
.
component
.
call_processing
.
schemes
.
ai_activity_type_request
import
AiActivityTypeRequest
11
+
from
src
.
component
.
call_processing
.
schemes
.
ai_activity_type_response
import
AiActivityTypeResponse
12...
|
Firefox
|
Jy 19798 evaluation for ai activity types by nikol Jy 19798 evaluation for ai activity types by nikolaybiaivanov · Pull Request #468 · jiminny/prophet — Work...
|
github.com/jiminny/prophet/pull/468/changes#diff-f github.com/jiminny/prophet/pull/468/changes#diff-fc8a653dec2cd8e3d632fa5d32e0abd4b22176ce7619cc5752c4efe9e53f083b...
|
8521
|
|
* Firefox File EditView History Bookmarks Profiles * Firefox File EditView History Bookmarks Profiles Tools Window HelpCPlatform Sprint 1 Q2 - Platform Tea( Jy 19798 evaluation for ai activ X+ New labI1 Open©github.com/jiminny/prophet/pull/468/changes#diff-fc8a653dec2cd8e3d632fa5d32e0abd4b22176ce7619cc5752c4efe9e53f083bJy 19798 evaluation for ai activity types #468All commitsnikolaybiaivanov wants to merge 9 commits into master from JY-19798-evaluation-for-ai-activity-types [v src/eval/eval-ai-activity-type.pyQ Filter files...• E eval_output/ai_activity_type/eval….> • ground_truth> D input• E src/evalendpoints.py* eval-ai-activity-type.pyE gitignore52 €6:2030 ll © | Daily - Platform • nowA100% C• 8• Tue 14 Apr 9:49:46• 1/ 203 viewed• Awallind aporovaliVlewedENDPOINT PATH = "call/ai-activity-tvoe"REQUEST_MODEL = AiActivityTypeRequestRESPONSE_MODEL = AiActivityTypeResponseOUTPUT_SUBFOLDER = "ai_activity_type"USES_LLM_EVALUATION = FalseREADS_FROM_DATASET_FOLDER = TrueDATA_SET: str = "initial"MINIMUM_REQUEST_BODY_SIZE: int = 0SKIP_UNREPLAYABLE_ON_COLLECT: bool = TrueREPLAYABILITY_CHECK_TIMEOUT_SECONDS: float = 20.0POPULATE_GROUND_TRUTH_LLM: bool = FalseGROUND_TRUTH_LLM_ENDPOINT: str = "OPENAI:gpt-5_2"COrPARE TO GROUND TRUTH: str = "es"def _init__(self, **kwargs: Any) -> None:super()._init_(**kwargs)_repo_root = Path(_file_).parents[2]_datasets_root = _repo_root / "eval_output" / self.OUTPUT_SUBFOLDER / "eval_datasets"_dataset_dir = _datasets_root / self._class_.DATA_SETself._dataset_dir =_dataset_dirself._inputs_dir =_dataset_dir / "input"self._ground_truths_dir =_dataset_dir / "ground_truth"async def collect_test_set(self) →> List[Dict[str, Any]l:if self.es_host_name:return awalt selt._collect_trom_est)return await self._collect_from_files() *async def _collect_from_es(self) → List[Dict[str, Anyll:if self. inputs dir.exists) or self. ground truths dir.exists):raise FileExistsError(f"Dataset folder already exists: {self._inputs_dir.parent).""Choose a different --data-set name or remove the folder manually."raw_samples = await es_collect_test_set(es_nost_name=self.es_host name.es_port=self.es_port,endpoint path=self.ENDPOINT PATH,Limit=self.limit,language_filter=self.language_filter,min_prompt_length=self.min_prompt_length,is_json_response=self.IS_JSON_RESPONSE,exclude_languages=self.exclude_languages,self._inputs_dir.mkdir(parents=True, exist_ok=True)...
|
NULL
|
NULL
|
NULL
|
8522
|
|
* Firefox File EditView History Bookmarks Profiles * Firefox File EditView History Bookmarks Profiles Tools Window HelpCPlatform Sprint 1 Q2 - Platform Tea( Jy 19798 evaluation for ai activ X+ New labI1 Open©github.com/jiminny/prophet/pull/468/changes#diff-fc8a653dec2cd8e3d632fa5d32e0abd4b22176ce7619cc5752c4efe9e53f083bJy 19798 evaluation for ai activity types #468All commitsnikolaybiaivanov wants to merge 9 commits into master from JY-19798-evaluation-for-ai-activity-types [v src/eval/eval-ai-activity-type.pyQ Filter files...• E eval_output/ai_activity_type/eval.…> D ground_truth> D input~ & src/evalendpoints.py* eval-ai-activity-type.pyE gitignore>0 lbl O | Daily - Platform • nowA100% C• 8• Tue 14 Apr 9:50:04• 1/ 203 viewed• Awallind aporovaliVomitrevew+330 UUUUU|• Viewedencboint vauneselT.eNDrUINi PAln.Ilimit=self.limit,language_filter=self.language_filter,min_prompt_length=self.min_prompt_length,is_json_response=self.IS_JSON_RESPONSE,exclude_languages=self.exclude_languages,[PASSWORD_DOTS] *+*++ +self._inputs_dir.mkdir(parents=True, exist_ok=True)self._ground_truths_dir.mkdir(parents=True, exist_ok=True)min size = self.MINIMUM REOUEST BODY SIZEsamples: List[Dict[str, Anyll = []skipped_no_response = 0skipped_too_small = 0skipped_unreplayable = 0skipped_probe_errors = 0endpoint_url = f"{self.api_host}/{self.ENDPOINT_PATH}"umeour = alontto.culentneour toraselt.KCPLAYABILIIY CHECK TIMEOUI SECUNDS)async with aiohttp.ClientSession(timeout=timeout) as session:for samole in raw samples:es_id = sample["id"]request boay = samplerequest bodyoriginal_response = sample.get("original_response")*if not original_response:skipped_no_response += 1continue1t man suze anc lencison,cunos.recvest oocv < min size.skipped_too_small += 11T SeLTaSALr UNKCrLATADLE UN LULLCCI:try:dsyne wiln sesszon.post enapone url,son-recuest boay as reso.1t resp.status = 200:body_text = await resp.text()iT"Lannor tino aculvlt in ooov text.skipped_unreplayable += 1except Exception:skipped probe errors += 1wiun ooen seut. 1nouls alr Ties 10r.50h, W dsTejson.dump(request_body, t, indent=2)with open(selt._ground_truths_dir / T"gt_tes_1dy•Json", "W") as T:json.dump(original_response, f, indent=2)samples.append(...
|
NULL
|
NULL
|
NULL
|
8523
|
|
JY-20543 add AJ reports User pilot tracking by Lak JY-20543 add AJ reports User pilot tracking by LakyLak · Pull Request #11932 · jiminny/app
Platform Sprint 1 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 1 Q2 - Platform Team - Scrum Board - Jira
Jy 19798 evaluation for ai activity types by nikolaybiaivanov · Pull Request #468 · jiminny/prophet
Jy 19798 evaluation for ai activity types by nikolaybiaivanov · Pull Request #468 · jiminny/prophet
Close tab
New Tab
Customize sidebar
Open Google Gemini (⌃X)
Tabs from other devices
Open history (⇧⌘H)
Open bookmarks (⌘B)
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
prophet
prophet
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
Issues
Pull requests
Repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Issues
Issues
Pull requests (6)
Pull requests
(
6
)
Agents
Agents
Actions
Actions
Projects
Projects
Wiki
Wiki
Security and quality (20)
Security and quality
(
20
)
Insights
Insights
Settings
Settings
Important update
Important update
On April 24 we'll start using GitHub Copilot interaction data for AI model training unless you opt out.
Review this update
Review this update
and manage your preferences in your
GitHub account settings
GitHub account settings
.
Dismiss banner
Jy 19798 evaluation for ai activity types #468 Edit title
Jy 19798 evaluation for ai activity types
#
468
Edit title
Preview
Preview
Awaiting approval
Awaiting approval
Code
Code
Open
nikolaybiaivanov
nikolaybiaivanov
wants to merge 9 commits into
master
master
from
JY-19798-evaluation-for-ai-activity-types
JY-19798-evaluation-for-ai-activity-types
Copy head branch name to clipboard
Lines changed: 25671 additions & 0 deletions
Conversation (3)
Conversation
(
3
)
Commits (9)
Commits
(
9
)
Checks (3)
Checks
(
3
)
Files changed (203)
Files changed
(
203
)
Pull Request Toolbar
Pull Request Toolbar
Collapse file tree
Open
Jy 19798 evaluation for ai activity types
Jy 19798 evaluation for ai activity types
#
468
All commits
All commits
nikolaybiaivanov
nikolaybiaivanov
wants to merge 9 commits into
master
master
from
JY-19798-evaluation-for-ai-activity-types
JY-19798-evaluation-for-ai-activity-types
Copy head branch name to clipboard
1
/
203
viewed
Awaiting approval
Awaiting approval
Submit review
Submit
review
Open diff view settings
Open overview panel
Open comments panel
(
0
)
Filter files…
Filter options
File tree
File tree
eval_output/ai_activity_type/eval_datasets/v2-from-logs-replayable
ground_truth
input
src/eval
endpoints.py
endpoints.py
eval-ai-activity-type.py
eval-ai-activity-type.py
.gitignore
.gitignore
This page has been optimized for large pull requests.
This page has been optimized for large pull requests.
This page has been optimized for large pull requests. Some browser features (like Find on Page or Select All) may not work as expected.
Switch to single file mode
Switch to single file mode
Collapse file
src/eval/eval-ai-activity-type.py
src/eval/eval-ai-activity-type.py
src/eval/eval-ai-activity-type.py
Copy file name to clipboard
Lines changed: 330 additions & 0 deletions
Not Viewed
Viewed
Comment on this file
More options
Original file line number
Original file line
Diff line number
Diff line change
@@ -0,0 +1,330 @@
1
+
#!/usr/bin/env python3
2
+
import
asyncio
3
+
import
json
4
+
import
os
5
+
from
pathlib
import
Path
6
+
from
typing
import
Any
,
Dict
,
List
7
+
8
+
import
aiohttp
9
+
10
+
from
src
.
component
.
call_processing
.
schemes
.
ai_activity_type_request
import
AiActivityTypeRequest
11
+
from
src
.
component
.
call_processing
.
schemes
.
ai_activity_type_response
import...
|
Firefox
|
Jy 19798 evaluation for ai activity types by nikol Jy 19798 evaluation for ai activity types by nikolaybiaivanov · Pull Request #468 · jiminny/prophet — Work...
|
github.com/jiminny/prophet/pull/468/changes#diff-f github.com/jiminny/prophet/pull/468/changes#diff-fc8a653dec2cd8e3d632fa5d32e0abd4b22176ce7619cc5752c4efe9e53f083b...
|
8524
|
|
JY-20543 add AJ reports User pilot tracking by Lak JY-20543 add AJ reports User pilot tracking by LakyLak · Pull Request #11932 · jiminny/app
Platform Sprint 1 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 1 Q2 - Platform Team - Scrum Board - Jira
Jy 19798 evaluation for ai activity types by nikolaybiaivanov · Pull Request #468 · jiminny/prophet
Jy 19798 evaluation for ai activity types by nikolaybiaivanov · Pull Request #468 · jiminny/prophet
Close tab
New Tab
Customize sidebar
Open Google Gemini (⌃X)
Tabs from other devices
Open history (⇧⌘H)
Open bookmarks (⌘B)
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
prophet
prophet
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
Issues
Pull requests
Repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Issues
Issues
Pull requests (6)
Pull requests
(
6
)
Agents
Agents
Actions
Actions
Projects
Projects
Wiki
Wiki
Security and quality (20)
Security and quality
(
20
)
Insights
Insights
Settings
Settings
Important update
Important update
On April 24 we'll start using GitHub Copilot interaction data for AI model training unless you opt out.
Review this update
Review this update
and manage your preferences in your
GitHub account settings
GitHub account settings
.
Dismiss banner
Jy 19798 evaluation for ai activity types #468 Edit title
Jy 19798 evaluation for ai activity types
#
468
Edit title
Preview
Preview
Awaiting approval
Awaiting approval
Code
Code
Open
nikolaybiaivanov
nikolaybiaivanov
wants to merge 9 commits into
master
master
from
JY-19798-evaluation-for-ai-activity-types
JY-19798-evaluation-for-ai-activity-types
Copy head branch name to clipboard
Lines changed: 25671 additions & 0 deletions
Conversation (3)
Conversation
(
3
)
Commits (9)
Commits
(
9
)
Checks (3)
Checks
(
3
)
Files changed (203)
Files changed
(
203
)
Pull Request Toolbar
Pull Request Toolbar
Collapse file tree
Open
Jy 19798 evaluation for ai activity types
Jy 19798 evaluation for ai activity types
#
468
All commits
All commits
nikolaybiaivanov
nikolaybiaivanov
wants to merge 9 commits into
master
master
from
JY-19798-evaluation-for-ai-activity-types
JY-19798-evaluation-for-ai-activity-types
Copy head branch name to clipboard
1
/
203
viewed
Awaiting approval
Awaiting approval
Submit review
Submit
review
Open diff view settings
Open overview panel
Open comments panel
(
0
)
Filter files…
Filter options
File tree
File tree
eval_output/ai_activity_type/eval_datasets/v2-from-logs-replayable
ground_truth
input
src/eval
endpoints.py
endpoints.py
eval-ai-activity-type.py
eval-ai-activity-type.py
.gitignore
.gitignore
This page has been optimized for large pull requests.
This page has been optimized for large pull requests.
This page has been optimized for large pull requests. Some browser features (like Find on Page or Select All) may not work as expected.
Switch to single file mode
Switch to single file mode
Collapse file
src/eval/eval-ai-activity-type.py
src/eval/eval-ai-activity-type.py
src/eval/eval-ai-activity-type.py
Copy file name to clipboard
Lines changed: 330 additions & 0 deletions
Not Viewed
Viewed
Comment on this file
More options
Original file line number
Original file line
Diff line number
Diff line change
@@ -0,0 +1,330 @@
1
+
#!/usr/bin/env python3
2
+
import
asyncio
3
+
import
json
4
+
import
os
5
+
from
pathlib
import
Path
6
+
from
typing
import
Any
,
Dict
,
List
7
+
8
+
import
aiohttp
9
+
10
+
from
src
.
component
.
call_processing
.
schemes
.
ai_activity_type_request
import
AiActivityTypeRequest
11
+
from
src
.
component
.
call_processing
.
schemes
.
ai_activity_type_response
import
AiActivityTypeResponse...
|
Firefox
|
Jy 19798 evaluation for ai activity types by nikol Jy 19798 evaluation for ai activity types by nikolaybiaivanov · Pull Request #468 · jiminny/prophet — Work...
|
github.com/jiminny/prophet/pull/468/changes#diff-f github.com/jiminny/prophet/pull/468/changes#diff-fc8a653dec2cd8e3d632fa5d32e0abd4b22176ce7619cc5752c4efe9e53f083b...
|
8525
|
|
DiaFilelEditViewTabsBookmarksHistoryExtensionsWind DiaFilelEditViewTabsBookmarksHistoryExtensionsWindowHelp$0.| Daily - Platform • nowmeet.google.com / Daily - PlatformNikolay Yankov (Presenting)Inbox (2) - niC 3y 19798 eval(7 Service-Deskjr Foet Ameson• Cost | Claude x©Actions secrC C DependatActions sec• Dependabot: xQ 8•Tue 14 Apr 9:50O Far Dependas xinny/prophet/actions/runs/24195639151/ob/70624720594?pr=468# AWS|E3 SSHс *El Datados * Clude5 CircieclD Sentry8 TaskslInsights & Coachin.• Dev• Ux0 Al BookmarksIEB)Summary|Run Flake8 & MypyRun details© UsageWorkfiow fileRun Flake8 & Mypy•, Search logs IStefka StoyanovaSet up Python> • Install Poetry |> • Load cached Poetry dependencies |> © Install dependencies using Poetry|v @ Run Flake81 •Run poetry run flakeß, -count -select-E9,F63,F7,F82,N293,N291,W292 --show-source -statistics./src/eval/eval-al-activity-type.py:330:63: N292 no newline at end of fileasyncio.runtrun_cvaluatzon(AlactavityTypetvaluator, args)Error: Process conpleted with exit code 1.• Run Mypy|• Post Load cached Poetry dependencies |• Post Set up Python|› © Post Checkout code> O Complete job• 2026 0i4u6, Inc.HortroPrivacy Security Status Community Docsvonest menesn25sSteliyan GeorgievKovalik9:50 AM | Daily - Platform4:498 • Tue 14 Apr 9:50:13ChatNikolay YankovNikolay Ivanov...
|
NULL
|
NULL
|
NULL
|
8526
|
|
Firefox FileEoitView History Bookmarks Profiles Firefox FileEoitView History Bookmarks Profiles Tools Window HelpCPlatform Sprint 1 Q2 - Platform Tea( Jy 19798 evaluation for ai activ X+ New labI1 Open©github.com/jiminny/prophet/pull/468/changes#diff-fc8a653dec2cd8e3d632fa5d32e0abd4b22176ce7619cc5752c4efe9e53f083bJy 19798 evaluation for ai activity types #468All commitsnikolaybiaivanov wants to merge 9 commits into master from JY-19798-evaluation-for-ai-activity-types [v src/eval/eval-ai-activity-type.pyQ Filter files...• E eval_output/ai_activity_type/eval….> • ground_truthinput.• E src/evalE endpoints.pyk* eval-ai-activity-type.pyE .gitignore>0 lbl O | Daily - Platform • nowA100% C• 8• Tue 14 Apr 9:50:13• 1/ 203 viewed• Awallino aporovaliVomitrevew+330 UUUUU• Viewed293294295296297299300301302304305306307308309310311312313314315316317318319321322323326328330+ if_name_ = "_main_":parser = create_parser(AiActivityTypeEvaluator)parser.add_argument("_-data-set",type=str,default="initial",dest="data_set",help="Dataset folder under eval output/ai activity_type/eval_datasets/ (derautt: initlal),parser.add_argument("--populate-ground-truth-llm",action="store_true",dest="populate_ground_truth_llm",parser.add_argument("--ground-truth-llm-endpoint",type=str,default="OPENAI : gpt-5_2",dest="ground_truth_llm_endpoint",parser.add_argument("--compare-to-ground-truth",type=str,choices=["es", "1lm"],default="es",dest="compare_to_ground_truth",args = parser.parse_args()AiActivityTypeEvaluator.DATA_SET = args.data_setAiActivityTypeEvaluator.POPULATE GROUND_TRUTH_LLM = bool(args.populate ground_truth_llmAiActivitvTvoeEvaluator.GROUND TRUTH LLM ENDPOINT = args.around truth 1lm endpointAiActivityTypeEvaluator.COMPARE_TO_GROUND_TRUTH = args.compare_to_ground_truthasyncio.run(run evaluation(AiActivityTypeEvaluator, arqs))> •gitignore @*+4 -uoov ViewedYou made it to the end....
|
NULL
|
NULL
|
NULL
|
8527
|
|
* Firefox FileEoitViewHistory BookmarksProfilesToo * Firefox FileEoitViewHistory BookmarksProfilesTools Window Help© github.com/jiminny/prophet/pull/468/changes#diff-d2f9fea9bc428b119ae26a66453e8eab7dd0782d4a4d93075ca5ce05903d77e811 OpenJyvoeVluar on Tor ai activty tvoes #408All commitsnikolaybiaivanov wants to merge 9 commits into master from JY-19798-evaluation-for-ai-activity-types [Platform Sprint 1 Q2 - Platform TeaQ Filter files...v src/eval/endpoints.py L *@ Jy 19798 evaluation for ai activ *+ New Tab• E eval_output/ai_activity_type/eval….> • ground_truth> L input.• E src/evalE endpoints.pynyD eval-ai-activity-type.pyE gitignore1721731741aa -172,6 +172,7 @@ async def process_entry(idx: int, entry: Dict[str, Any]) - None:known_keys ={"content", "summary", "themes", "action_items","salesRep", "customer",176177is_empty = known_keys.isdisjoint(new_data.keys())@ -212,6 +213,10 @ async def process_entry(idx: int, entry: Dict[str, Anyl) -> None:2122132141"original_prompt": entry.get("original_prompt"),"current prompt": current prompt,"ground_truth": entry.get("ground_truth"),215216217"new_response": new_data,"valid_schema": False,"response_time": response_time,v src/eval/eval-ai-activity-type.py0c -0,0 +1,330 gg• | Daily - Platform • nowA100% CD8 • Tue 14 Apr 9:50:14^ 1 / 203 viewed• Awallind aporovalito uuuuu• Viewed172173174175 +1761771/8213214215216 [PHONE]21222known_keys = {"content", "summary", "themes", "action_items","saleskep", "customer","ai_activity_type", "ai_activity_type_reasoning",is_empty = known_keys.isdisjoint(new_data.keys())"original_prompt": entry.get("original_prompt"),"current_prompt": current_prompt,"ground_truth": entry.get("ground_truth"),"ground_truth_es": entry.get ("ground_truth_es"),"ground_truth_llm": entry.get("ground_truth_llm"),"ground_truth_llm_model": entry.get("ground_truth_llm_model"),"ground_truch_tum_reasoning: entry.gett ground_truch_lum_reasoning),"new_response": new_data,"valid_schema": ralse,"response_time"; response_time,+330 UUU Viewed+ #!/usr/bin/env python3+ import asyncio+ import json+Import os+ from pathlib import Path*Tron Lyoino 1mporl Any, UIcl, L15+ Import dloneuo+ from src.component.call_processing.schemes.ai activity_ type_request import AiActivityTypeRequest+Tron src,conbonent.call orocessino.schenes.al actlviry tvoe resconse 1nbort AlAculVitV viekesconse+ from src.eval.abstract_evaluator import AbstractEvaluator+ from src.eval.collectors import collect test set as es collect test set+ from src.eval.common_evaluator_runner import create_parser, run_evaluation+ from src.eval.endpoints import run endpoint as run endpoint http+ trom src.eval.models import EvaluationResult+ class AiActivityTypeEvaluator(AbstractEvaluator):ENDPUINI_PAIN = "call/al-activity-type"REQUEST_MODEL = AiActivityTypeRequestRESPONSE MODEL = AiActivitvTvpeResponseOUTPUT_SUBFOLDER = "ai_activity_type"USES_LLM_EVALUATION = FalseREADS_FROM_DATASET_FOLDER = TrueDATA_SET: str = "initial"...
|
NULL
|
NULL
|
NULL
|
8528
|
|
JY-20543 add AJ reports User pilot tracking by Lak JY-20543 add AJ reports User pilot tracking by LakyLak · Pull Request #11932 · jiminny/app
Platform Sprint 1 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 1 Q2 - Platform Team - Scrum Board - Jira
Jy 19798 evaluation for ai activity types by nikolaybiaivanov · Pull Request #468 · jiminny/prophet
Jy 19798 evaluation for ai activity types by nikolaybiaivanov · Pull Request #468 · jiminny/prophet
Close tab
New Tab
Customize sidebar
Open Google Gemini (⌃X)
Tabs from other devices
Open history (⇧⌘H)
Open bookmarks (⌘B)
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
prophet
prophet
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
Issues
Pull requests
Repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Issues
Issues
Pull requests (6)
Pull requests
(
6
)
Agents
Agents
Actions
Actions
Projects
Projects
Wiki
Wiki
Security and quality (20)
Security and quality
(
20
)
Insights
Insights
Settings
Settings
Important update
Important update
On April 24 we'll start using GitHub Copilot interaction data for AI model training unless you opt out.
Review this update
Review this update
and manage your preferences in your
GitHub account settings
GitHub account settings
.
Dismiss banner
Jy 19798 evaluation for ai activity types #468 Edit title
Jy 19798 evaluation for ai activity types
#
468
Edit title
Preview
Preview
Awaiting approval
Awaiting approval
Code
Code
Open
nikolaybiaivanov
nikolaybiaivanov
wants to merge 9 commits into
master
master
from
JY-19798-evaluation-for-ai-activity-types
JY-19798-evaluation-for-ai-activity-types
Copy head branch name to clipboard
Lines changed: 25671 additions & 0 deletions
Conversation (3)
Conversation
(
3
)
Commits (9)
Commits
(
9
)
Checks (3)
Checks
(
3
)
Files changed (203)
Files changed
(
203
)
Pull Request Toolbar
Pull Request Toolbar
Collapse file tree
Open
Jy 19798 evaluation for ai activity types
Jy 19798 evaluation for ai activity types
#
468
All commits
All commits
nikolaybiaivanov
nikolaybiaivanov
wants to merge 9 commits into
master
master
from
JY-19798-evaluation-for-ai-activity-types
JY-19798-evaluation-for-ai-activity-types
Copy head branch name to clipboard
1
/
203
viewed
Awaiting approval
Awaiting approval
Submit review
Submit
review
Open diff view settings
Open overview panel
Open comments panel
(
0
)
Filter files…
Filter options
File tree
File tree
eval_output/ai_activity_type/eval_datasets/v2-from-logs-replayable
ground_truth
input
src/eval
endpoints.py
endpoints.py
eval-ai-activity-type.py
eval-ai-activity-type.py
.gitignore
.gitignore
This page has been optimized for large pull requests.
This page has been optimized for large pull requests.
This page has been optimized for large pull requests. Some browser features (like Find on Page or Select All) may not work as expected.
Switch to single file mode
Switch to single file mode
Collapse file...
|
Firefox
|
Jy 19798 evaluation for ai activity types by nikol Jy 19798 evaluation for ai activity types by nikolaybiaivanov · Pull Request #468 · jiminny/prophet — Work...
|
github.com/jiminny/prophet/pull/468/changes#diff-d github.com/jiminny/prophet/pull/468/changes#diff-d2f9fea9bc428b119ae26a66453e8eab7dd0782d4a4d93075ca5ce05903d77e8...
|
8529
|
|
JY-20543 add AJ reports User pilot tracking by Lak JY-20543 add AJ reports User pilot tracking by LakyLak · Pull Request #11932 · jiminny/app
Platform Sprint 1 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 1 Q2 - Platform Team - Scrum Board - Jira
Jy 19798 evaluation for ai activity types by nikolaybiaivanov · Pull Request #468 · jiminny/prophet
Jy 19798 evaluation for ai activity types by nikolaybiaivanov · Pull Request #468 · jiminny/prophet
Close tab
New Tab
Customize sidebar
Open Google Gemini (⌃X)
Tabs from other devices
Open history (⇧⌘H)
Open bookmarks (⌘B)
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
prophet
prophet
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
Issues
Pull requests
Repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Issues
Issues
Pull requests (6)
Pull requests
(
6
)
Agents
Agents
Actions
Actions
Projects
Projects
Wiki
Wiki
Security and quality (20)
Security and quality
(
20
)
Insights
Insights
Settings
Settings
Important update
Important update
On April 24 we'll start using GitHub Copilot interaction data for AI model training unless you opt out.
Review this update
Review this update
and manage your preferences in your
GitHub account settings
GitHub account settings
.
Dismiss banner
Jy 19798 evaluation for ai activity types #468 Edit title
Jy 19798 evaluation for ai activity types
#
468
Edit title
Preview
Preview
Awaiting approval
Awaiting approval
Code
Code
Open
nikolaybiaivanov
nikolaybiaivanov
wants to merge 9 commits into
master
master
from
JY-19798-evaluation-for-ai-activity-types
JY-19798-evaluation-for-ai-activity-types
Copy head branch name to clipboard
Lines changed: 25671 additions & 0 deletions
Conversation (3)
Conversation
(
3
)
Commits (9)
Commits
(
9
)
Checks (3)
Checks
(
3
)
Files changed (203)
Files changed
(
203
)
Pull Request Toolbar
Pull Request Toolbar
Collapse file tree
Open
Jy 19798 evaluation for ai activity types
Jy 19798 evaluation for ai activity types
#
468
All commits
All commits
nikolaybiaivanov
nikolaybiaivanov
wants to merge 9 commits into
master
master
from
JY-19798-evaluation-for-ai-activity-types
JY-19798-evaluation-for-ai-activity-types
Copy head branch name to clipboard
2
/
203
viewed
Awaiting approval
Awaiting approval
Submit review
Submit
review
Open diff view settings
Open overview panel
Open comments panel
(
0
)
Filter files…
Filter options
File tree
File tree
eval_output/ai_activity_type/eval_datasets/v2-from-logs-replayable
ground_truth
input
src/eval
endpoints.py
endpoints.py
eval-ai-activity-type.py
eval-ai-activity-type.py...
|
Firefox
|
Jy 19798 evaluation for ai activity types by nikol Jy 19798 evaluation for ai activity types by nikolaybiaivanov · Pull Request #468 · jiminny/prophet — Work...
|
github.com/jiminny/prophet/pull/468/changes#diff-d github.com/jiminny/prophet/pull/468/changes#diff-d2f9fea9bc428b119ae26a66453e8eab7dd0782d4a4d93075ca5ce05903d77e8...
|
8530
|
|
Firefox File EditView History Bookmarks ProfilesT Firefox File EditView History Bookmarks ProfilesTools Window Help•..•+CPlatform Sprint 1 Q2 - Platform Tea@ Jy 19798 evaluation for ai activ *+ New lab© github.com/jiminny/prophet/pull/468/changes#diff-d2f9fea9bc428b119ae26a66453e8eab7dd0782d4a4d93075ca5ce05903d77e8i? OpenJyvoeValuat on Tor ai activity tvoes ·400All commitsnikolaybiaivanov wants to merge 9 commits into master from JY-19798-evaluation-for-ai-activity-types [Q Filter files...› src/eval/endpoints.py 0 *• E eval_output/ai_activity_type/eval….> • ground_truth> E input.• E src/evalendpoints.pyD eval-ai-activity-type.py*.gitignorev src/eval/eval-ai-activity-type.py*• @a -0,0 +1,330 @dlobl1 Daily - Platform • nowA100% C•8 • Tue 14 Apr 9:50:22^ 2 1 203 viewed• Awallind aporovaliSuomitrevewto+330 mViewed• Viewed+*!usrdin, env prtnons+ import asyncio+ 1mporl son4 + I import os+ from pathlib import Path+ Trom typing import Any, bict, List+ Import alonuto.+ from src.component.call_processing.schemes.ai activity_type_request import AlActivitylyperequest+ from src.component.call_processing.schemes.ai_activity_type_response import AiActivityTypeResponse+ from src.eval.abstract evaluator import AbstractEvaluator+ from src.eval.collectors import collect_test_set as es_collect_test_set+ from src.eval.common_evaluator_runner import create_ parser, run_evaluation+ from src.eval.endpoints import run_endpoint as run_endpoint_http+ from src.eval.models import EvaluationResult203+ class AiActivityTypeEvaluator(AbstractEvaluator):ENDPOINT_PATH = "call/ai-activity-type"REQUEST_MODEL = AiActivityTypeRequestRESPONSE_MODEL = AiActivityTypeResponseOUTPUT_SUBFOLDER = "ai_activity_type"USES_LLM_EVALUATION = FalseREADS_FROM_DATASET_FOLDER = TrueDATA_SET: str = "initial"MINIMUM_REQUEST_BODY_SIZE: int = 0SKIP_UNREPLAYABLE_ON_COLLECT: bool = TrueREPLAYABILITY CHECK TIMEOUT SECONDS: float = 20.0POPULATE GROUND TRUTH LLM: bool = FalseGROUND_TRUTH_LLM_ENDPOINT: str = "'OPENAI: gpt-5_2"COMPARE_TO_GROUND_TRUTH: str = "es"def _init__(self, **kwargs: Any) -> None:super()._init_(**kwargs)_repo_root = Path(_file_).parents(2]_datasets_root = _repo_root / "eval_output" / self.OUTPUT_SUBFOLDER / "eval_datasets"_dataset_dir = _datasets_root / self._class_.DATA_SETself._dataset_dir = _dataset_dirself._inputs_dir = _dataset_dir / "input"self._ground_truths_dir = _dataset_dir / "ground_truth"async def collect_test_set(self) -> List[Dict[str, Anyll:...
|
NULL
|
NULL
|
NULL
|
8531
|
|
JY-20543 add AJ reports User pilot tracking by Lak JY-20543 add AJ reports User pilot tracking by LakyLak · Pull Request #11932 · jiminny/app
Platform Sprint 1 Q2 - Platform Team - Scrum Board - Jira
Platform Sprint 1 Q2 - Platform Team - Scrum Board - Jira
Jy 19798 evaluation for ai activity types by nikolaybiaivanov · Pull Request #468 · jiminny/prophet
Jy 19798 evaluation for ai activity types by nikolaybiaivanov · Pull Request #468 · jiminny/prophet
Close tab
New Tab
Customize sidebar
Open Google Gemini (⌃X)
Tabs from other devices
Open history (⇧⌘H)
Open bookmarks (⌘B)
Skip to content
Skip to content
Open menu
Homepage (g then d)
jiminny
jiminny
prophet
prophet
Search or jump to…
Type
/
to search
Chat with Copilot
Open Copilot…
Create new...
Issues
Pull requests
Repositories
You have unread notifications(g then n)
Open user navigation menu
Repository navigation
Repository navigation
Code
Code
Issues
Issues
Pull requests (6)
Pull requests
(
6
)
Agents
Agents
Actions
Actions
Projects
Projects
Wiki
Wiki
Security and quality (20)
Security and quality
(
20
)
Insights
Insights
Settings
Settings
Important update
Important update
On April 24 we'll start using GitHub Copilot interaction data for AI model training unless you opt out.
Review this update
Review this update
and manage your preferences in your
GitHub account settings
GitHub account settings
.
Dismiss banner
Jy 19798 evaluation for ai activity types #468 Edit title
Jy 19798 evaluation for ai activity types
#
468
Edit title
Preview
Preview
Awaiting approval
Awaiting approval
Code
Code
Open...
|
Firefox
|
Jy 19798 evaluation for ai activity types by nikol Jy 19798 evaluation for ai activity types by nikolaybiaivanov · Pull Request #468 · jiminny/prophet — Work...
|
github.com/jiminny/prophet/pull/468/changes#diff-d github.com/jiminny/prophet/pull/468/changes#diff-d2f9fea9bc428b119ae26a66453e8eab7dd0782d4a4d93075ca5ce05903d77e8...
|
8532
|
|
Firefox File Edit View History Bookmarks Profiles Firefox File Edit View History Bookmarks Profiles Tools Window Help••+→ cPlatform Sprint 1 Q2 - Platform Tea( Jy 19798 evaluation for ai activ X+ New labi? Open© github.com/jiminny/prophet/pull/468/changes#diff-d2f9fea9bc428b119ae26a66453e8eab7dd0782d4a4d93075ca5ce05903d77e8Jy 19798 evaluation for ai activity types #468All commitsnikolaybiaivanov wants to merge 9 commits into master from JY-19798-evaluation-for-ai-activity-types> src/eval/endpoints.py 0 +Q Filter files...• E eval_output/ai_activity_type/eval….> D ground_truth>B input• E src/evalendpoints.pyD eval-ai-activity-type.py*.gitignorev src/eval/eval-ai-activity-type.py0c -0,0 +1,330 gg24•< 4o ll • | Daily - Platform • nowA100% C• 8 • Tue 14 Apr 9:50:28^ 2 1 203 viewed• Awallind aporovaliSuomitrevewy+5 -n Viewed+330 UUU Viewed+ #!/usr/bin/env python3|+ Import async1o+ import json+Import os+ Trom pathlib import Path+ Trom Lyoino 1mporl Any, UIcl L150+ Import dlonuuo+ from src.component.call_processing.schemes.ai activity_ type_request import AiActivityTypeRequest+ Trom src.component.call_processing.schemes.al_activity-type_response import Alactivitylyperesponse+ from src.eval.abstract_evaluator import AbstractEvaluator+ Trom src.eval.couleccors imporr collecc tesc ser as es collect test seu+ from src.eval.common_evaluator_runner import create_parser, run_evaluation+ from src.eval.endpoints import run_endpoint as run_endpoint_http+ trom src.eval.models import EvaluationResult+ class AiActivityTypeEvaluator(AbstractEvaluator):ENDPOINT_PATH = "call/ai-activity-type"REQUEST_MODEL = AiActivityTypeRequestRESPONSE_MODEL = AiActivityTypeResponseOUTPUT_SUBFOLDER = "ai_activity_type"USES_LLM_EVALUATION = FalseREADS_FROM_DATASET_FOLDER = TrueDATA_SET: str = "initial"MINIMUM_REQUEST_BODY_SIZE: int = 0SKIP_UNREPLAYABLE_ON_COLLECT: bool = TrueREPLAYABILITY_CHECK_TIMEOUT_SECONDS: float = 20.0POPULATE_GROUND_TRUTH_LLM: bool = FalseGROUND_TRUTH_LLM_ENDPOINT: str = "OPENAI:gpt-5_2"COMPARE_TO_GROUND_TRUTH: str = "es"def _init_(self, **kwargs: Any) -> None:super()._init_(**kwargs)_repo_root = Path(_file_).parents[2]_datasets_root = _repo_root / "eval_output" / self.OUTPUT_SUBFOLDER / "eval_datasets"_dataset_dir = _datasets_root / self._class_.DATA_SETself._dataset_dir =_dataset_dirself._inputs_dir = _dataset_dir / "input"self._ground_truths_dir = _dataset_dir / "ground_truth"async der coltect_test_seu(seut) → Listibicuistr, AnyJu:if self.es host name:...
|
NULL
|
NULL
|
NULL
|
8533
|
|
FirefoxFileEoitViewHistoryBookmarksProfilesToolsWi FirefoxFileEoitViewHistoryBookmarksProfilesToolsWindowHelpgithub.com/fiminny/prophet/pull/468/changes#diff-d2f9fea9bc428b119ae26a66453e8eab/dd0782d4a4d93075ca5ce05903d77e8Google GeminiJy 19798 evaluation for ai activity tvpes #468Platform Sprint 1 Q2 - Platform Tea( Jy 19798 evaluation for ai activ+ New lab8? OperAll commits-nikolaybiaivanov wants to merge 9 commits into master from JY-19798-evaluation-for-ai-activity-types Lsrc/eval/endpoints.pyE• Filter files...v = eval_ output/ai_activity_type/eval...v src/eval/eval-ai-activity-type.py L> • ground_truth..•aa-0.0 +1.330 ad>I inoutv F src/eval#endpoints.py+ eval-ai-activity-type.pyE .gitignoreDaily - Platform • now100%• Tue 14 Apr 9:50:29• 2 | 203 viewed|Awallind aporovali45 nannOI• Viewed+330 UUUUuVlewed+ #!/usr/bin/env python3+ 1mport asynclo+ import json+ import os+ trom pathlib import Path+ from typing import Any, Dict, List+ import aiohttp+ from src.component.call_processing.schemes.ai activity type_request importALACELVLEV voekequest11+ from src.component.call_processing.schemes.ai activity type_response importA1ACCIVICYIypekesponse+ trom src.eval.abstract_evaluator import Abstractevaluator+ from src.eval.collectors import collect test set as es collect test set+ trom src.eval.common_evaluator_runner import create_parser, run_evaluation+ from src.eval.endpoints import run endpoint as run endpoint http+ trom src,eval,models 1moort Evaluatzonkesult23Ask GeminiNewsummarize page+ class AiActivityTypeEvaluator(AbstractEvaluator):ENDPOINT PATH = "call/ai-activity-type"REQUEST_MODEL = AiActivityTypeRequestRESrUNst MoDEL = ALACCIviLyIvpekesponseouTPUT_sUBFOLDER = "a1_activity_type"USES LLM EVALUATION = FalseREADS_FROM_DATASET_ FOLDER = TrueDATA SET: str = "initial"MINIMUM_REQUEST BODY_SIZE: int = 0SKLP_UNREPLAYABLE_ON_COLLECT: bool = TrueREPLAYABILITY CHECK TIMEOUT SECONDS: float = 20.0rurULAlL onuuNu Ikuin LlM. Dool = IalseGROUND TRUTH LIM ENDPOINT: str = "OPENAL:apt=5 2"LUMPAKE TU GKUUND IKuin: Str=esdeт 1nic selt, **kwaros. Any -> None:super().__init__(**kwargs)_repo_root = Path(_file_).parents (2]_datasets_root = _repo_root / "eval_output" / selt.OUTPUT_SUBFOLDER /"eval datasets"_dataset_dir = _datasets_root / selt.__class_.DATA_SElself._dataset_dir =_dataset_dirselt._inputs_air =_dataset_air / "Input"self. around truths dir= datacet dir "around truth'l...
|
NULL
|
NULL
|
NULL
|
8534
|
|
DiaFilelEditViewTabsBookmarksHistoryExtensionsWind DiaFilelEditViewTabsBookmarksHistoryExtensionsWindowHelp$0.| Daily - Platform • nowmeet.google.com / Daily - PlatformNikolay Yankov (Presenting)Inbox (2) - niE:1Q 8•Tue 14 Apr 9:50C 3y 19798 eval(7 Service-Deskjr Foet Ameson• Cost | Claude x©Actions secrC C DependatActions sec• Dependabot: xHa Dependatinny/prophet/actions/runs/24195639151/ob/70624720594?pr=468B AWSE3 SSHс *El Datados * Clude5 CircieclD Sentry8 TaskslInsights & Coachin.• Dev• Ux.0 Al BookmarksRun Flake8 & Mypy•, Search logs ISummary|Run Flake8 & MypyRun details© UsageWorkfiow fileStefka StoyanovalHDSet up Python> O Install Poetry|> • Load cached Poetry dependencies |> © Install dependencies using Poetry|v @ Run Flake81 •Run poetry run flakeß, -count -select-E9,F63,F7,F82,N293,N291,W292 --show-source -statistics./src/eval/eval-al-activity-type.py:330:63: N292 no newline at end of fileasyncio.runtrun_cvaluatzon(AlactavityTypetvaluator, args)Error: Process conpleted with exit code 1.• Run Mypy|• Post Load cached Poetry dependencies |• Post Set up Python |› © Post Checkout code> O Complete job• 2026 0i4u6, Inc.HortroPrivacy Security Status Community Docsvonest menesn25s7sSteliyan GeorgievKovalik9:50 AM | Daily - Platform5:088 • Tue 14 Apr 9:50:32Chat6Nikolay YankovNikolay Ivanov...
|
NULL
|
NULL
|
NULL
|
8535
|
|
FirefoxFileEoitViewHistoryBookmarksProfilesToolsWi FirefoxFileEoitViewHistoryBookmarksProfilesToolsWindow Helpgithub.com/jiminny/prophet/pull/468/changes#diff-d2f9fea9bc428b119ae26a66453e8eab7dd0782d4a4d93075ca5ce05903d77e8~ Google Gemini...l1 OpenJy 19798 evaluation for ai activity types #468All commitsnikolaybiaivanov wants to merge 9 commits into master from JY-19798-evaluation-for-ai-activity-types LPlatform Sprint 1 Q2 - Platform TeaGeminiPROsrc/eval/endpoints.pyEQ Filter files...( Jy 19798 evaluation for ai activ+ New lab• E eval_output/ai_activity_type/eval.….> • ground_truth>I inoutv src/eval/eval-ai-activity-type.py..•0a -0,0 +1,330 ggE src/evalE endpoints.pyE eval-ai-activity-type.py#.gitignore1 Daily - Platform • nowA• 2 1 203 viewed|• Awallind aporovall100%8 • Tue 14 Apr 9:50:3145 П0OOOT• Viewed+330 UuuuuVlewedHi LukasWhere should westart?Create image* Create musicBoost my dayHelp me learnWrite anythingO Enter a prompt for Gemini+New]summarise pagePro v+ #!/usr/bin/env python3+ 1mport asynclo+ import json+ import os+ from pathlib import Path+ from typing import Any, Dict, List+ import aiohttp+ from src.component.call_processing.schemes.ai activity type_request importALACELVLEV voekequest11+ from src.component.call_processing.schemes.ai_activity_type_response importA1ACCIVICYIypekesponse+ from src.eval.abstract_evaluator import AbstractEvaluator+ from src.eval.collectors import collect test set as es collect test set+ from src.eval.common_evaluator_runner import create_parser, run_evaluation+ from src.eval.endpoints import run endpoint as run endpoint http+ trom src,eval,models 1moort Evaluatzonkesult23+ class AiActivityTypeEvaluator(AbstractEvaluator):ENDPOINT_PATH = "call/ai-activity-type"REQUEST_MODEL = AiActivityTypeRequestRESrUNst MoDEL = ALACCIviLyIvpekesponseOUTPUT_SUBFOLDER = "ai_activity_type"USES_LLM_EVALUATION = FalseREADS_FROM_DATASET_FOLDER = TrueDATA_SET: str = "initial"MINIMUM_REQUEST_BODY_SIZE: int = 0SKIP_UNREPLAYABLE_ON_COLLECT: bool = TrueREPLAYABILITY_CHECK_TIMEOUT_SECONDS: float = 20.0rurULAlL onuuNu Ikuin LlM. Dool = IalseGROUND TRUTH LIM ENDPOINT: str = "OPENAL:apt=5 2"LUMPAKE TO GKUUND TKUin: Str=esdeт 1nic selt, **kwaros. Any -> None:super()._init_(**kwargs)_repo_root = Path(_file_).parents (2]_datasets_root =_repo_root / "eval_output" / self.OUTPUT_SUBFOLDER /"eval datasets"_dataset_dir = _datasets_root / self._class_.DATA_SETself._dataset_dir =_dataset_dirselt._inputs_air = _dataset_dar / "input"self. around truths dir = dataset dir f "around +ruth"...
|
NULL
|
NULL
|
NULL
|
8536
|