Date: Fri, 29 Mar 2024 08:36:53 +0000 (UTC) Message-ID: <2004957574.7713.1711701413866@ip-10-10-7-29.ec2.internal> Subject: Exported From Confluence MIME-Version: 1.0 Content-Type: multipart/related; boundary="----=_Part_7712_1485844404.1711701413865" ------=_Part_7712_1485844404.1711701413865 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Content-Location: file:///C:/exported.html
The goal of this project is to centrally collect TIER data about Grouper= deployment to help improve Grouper and give information to TIER constituen= ts about Grouper usage. As of Grouper 2.3 Patch 62, no information is= automatically sent to TIER as we are waiting for TIER to have a production= collector. However, the Grouper UI has a new feature that allows loc= al administrators to view some of the data that would be sent to TIER.
A new thread runs in each JVM process that keeps track of updates being = made of various types:
The thread will update counts of these various types to the Grouper data= base every 1 hour by default (configured via grouper.properties - inst= rumentation.updateGrouperIntervalInSeconds).
Another configuration option will specify the increments to keep counts = of (instrumentation.updateIncrements). E.g. if we're keeping counts b= y 10 minutes or hour or day. This also defaults to 1 hour.
This data in Grouper is kept for 30 days but configurable (instrumentati= on.retainData.days).
Grouper administrators can view the counts gathered by the instrumentati= on thread in the Grouper UI. Other users can also be given access by = configuring uiV2.admin.instrumentation.must.be.in.group.
Under Quick Links, click on Miscellaneous and then click on In= strumentation.
At the top, it will show you each of your Grouper instances (currently D= aemon, UI, and WS).
The Last update column indicates the last time that the instrumentation = thread for that instance updated the database with its counts.
Below that, you can view the overall counts for the various types.
The counts are displayed by day. If you select a specific day, you= can view hourly counts for that day.
Also, you can view the same data for each individual instance by clickin= g on the instance UUID.
Note this will go to a test collector. Shouldn't be used in produc= tion at the moment.
Get patches for 2.3 (24 and 25)
Set this in grouper-loader.properties
otherJo= b.tierInstrumentationDaemon.class =3D edu.internet2.middleware.grouper.inst= rumentation.TierInstrumentationDaemon otherJob.tierInstrumentationDaemon.quartzCron =3D 0 0 2 * * ?
|
{ serviceEnabled: true, endpoints: [ { uri: "https://grouperdemo.internet2.edu/tierInstrumentationCollector= /tierInstrumentationCollector/v1/upload" }, { uri: "https://grouperdemo2.internet2.edu/tierInstrumentationCollecto= r2/tierInstrumentationCollector/v1/upload" } ] }
{ "reportFormat":1, "component":"grouper", "institution=E2=80=9D:=E2=80=9DPenn=E2=80=9D, "environment=E2=80=9D:=E2=80=9Dprod=E2=80=9D, "version":"2.3.0", "platformWindows":false, "platformLinux":false, "platformMac":true, "platformSolaris":false, "transactionCountMemberships":6, "transactionCountPrivileges":0, "registryCountDirectMemberships":2, "registryCountDirectPrivileges":54, "registryCountDirectPermissions":0, "provisionToLdapUsingPsp":false, "provisionToLdapUsingPspng":false, "patchesInstalled":[ =20 ], "instances":[ =20 { =20 "uuid":"d0a363444b49489591b80690a6edc09d", "engineName":"grouperShell", "serverLabel":"-2", "lastUpdate":1489762214975, "newCounts":[ =20 ] }, { =20 "uuid":"2898de277109417ebccd00cee00f0649", "engineName":"grouperLoader", "serverLabel":"-2", "lastUpdate":1489769384218, "newCounts":[ =20 ] }, { =20 "uuid":"1a8885db0b564e94b3958dbfc7b032f0", "engineName":"grouperUI", "serverLabel":"-2", "lastUpdate":1489769384725, "newCounts":[ =20 { =20 "startTime":1489767550000, "duration":5000, "UI_REQUESTS":5 }, { =20 "startTime":1489767600000, "duration":5000, "UI_REQUESTS":1 }, { =20 "startTime":1489767635000, "duration":5000, "UI_REQUESTS":1 }, { =20 "startTime":1489767650000, "duration":5000, "UI_REQUESTS":1 }, { =20 "startTime":1489767660000, "duration":5000, "API_GROUP_ADD":1, "UI_REQUESTS":2 }, { =20 "startTime":1489767690000, "duration":5000, "API_GROUP_DELETE":1, "UI_REQUESTS":3 }, { =20 "startTime":1489767720000, "duration":5000, "UI_REQUESTS":1 }, { =20 "startTime":1489767725000, "duration":5000, "UI_REQUESTS":3 }, { =20 "startTime":1489767730000, "duration":5000, "UI_REQUESTS":3, "API_STEM_ADD":1 }, { =20 "startTime":1489767735000, "duration":5000, "UI_REQUESTS":2, "API_STEM_DELETE":1 }, { =20 "startTime":1489767770000, "duration":5000, "UI_REQUESTS":1 }, { =20 "startTime":1489767790000, "duration":5000, "UI_REQUESTS":2 }, { =20 "startTime":1489767795000, "duration":5000, "UI_REQUESTS":2, "API_GROUP_ADD":1 }, { =20 "startTime":1489767800000, "duration":5000, "UI_REQUESTS":1 }, { =20 "startTime":1489767805000, "duration":5000, "API_MEMBERSHIP_ADD":1, "UI_REQUESTS":3 }, { =20 "startTime":1489767835000, "duration":5000, "UI_REQUESTS":1 }, { =20 "startTime":1489769030000, "duration":5000, "UI_REQUESTS":4 }, { =20 "startTime":1489769035000, "duration":5000, "UI_REQUESTS":1 }, { =20 "startTime":1489769045000, "duration":5000, "UI_REQUESTS":4 }, { =20 "startTime":1489769170000, "duration":5000, "UI_REQUESTS":1 }, { =20 "startTime":1489769175000, "duration":5000, "UI_REQUESTS":2 }, { =20 "startTime":1489769275000, "duration":5000, "API_MEMBERSHIP_DELETE":1, "UI_REQUESTS":1 }, { =20 "startTime":1489769315000, "duration":5000, "API_MEMBERSHIP_ADD":1, "UI_REQUESTS":4 } ] } ] }
The value of the assignment on the assignment (instrumentationDataCo= unts) will be like:
{"start= Time" : 1486753200000, "duration" : 600000, "UI_REQUESTS" : 30, "API_GROUP_= ADD" : 5, "API_GROUP_DELETE" : 3}
There may be multiple values added each time it runs. For exam= ple, if the database is updated every hour and the increment is every 10 mi= nutes, then it could add 6 of these.
{"start= Time" : 1486753200000, "duration" : 600000, "UI_REQUESTS" : 30, "API_GROUP_= ADD" : 5, "API_GROUP_DELETE" : 3} {"startTime" : 1486753800000, "duration" : 600000, "UI_REQUESTS" : 300, "AP= I_GROUP_ADD" : 2, "API_GROUP_DELETE" : 6} {"startTime" : 1486754400000, "duration" : 600000, "UI_REQUESTS" : 3000, "A= PI_GROUP_ADD" : 1, "API_GROUP_DELETE" : 2}etc
The TIER instrumentation daemon will sends these to TIER.
"instan= ces" : [ { "uuid" : "uuid1",=20 "engineName" : "grouperUI",=20 "serverLabel" : "ui-01" "lastUpdate" : 1488825739828,=20 "newCounts" : [{"startTime" : 1486753200000, "duration" := 600000, "UI_REQUESTS" : 30, "API_GROUP_ADD" : 5, "API_GROUP_DELETE" : 3},= =20 {"startTime" : 1486753800000, "duration" := 600000, "UI_REQUESTS" : 300, "API_GROUP_ADD" : 2, "API_GROUP_DELETE" : 6},= =20 {"startTime" : 1486754400000, "duration" := 600000, "UI_REQUESTS" : 3000, "API_GROUP_ADD" : 1, "API_GROUP_DELETE" : 2}= ]=20 },=20 { "uuid" : "uuid2",=20 "serverLabel" : "ui-02" "engineName" : "grouperUI",=20 "lastUpdate" : 1488825739829 }, { "uuid" : "uuid3", "serverLabel" : "ws-01" "engineName" : "grouperWS", "lastUpdate" : 1488825739829 }, { "uuid" : "uuid4", "serverLabel" : "ws-02" "engineName" : "grouperWS", "lastUpdate" : 1488825739829 }, { "uuid" : "uuid5", "serverLabel" : "daemon-01" "engineName" : "grouperLoader", "lastUpdate" : 1488825739829 } ]
An attribute will be created for each collector (e.g. etc:attri= bute:instrumentationData:instrumentationDataCollectors:OTHER_JOB_tierInstru= mentationDaemon). This will be assigned to another group (etc:attribu= te:instrumentationData:instrumentationDataCollectorsGroup). And that = assignment will have the time the collector was last updated (etc:attribute= :instrumentationData:instrumentationDataCollectorLastUpdate).
The values won't be audited (user audit or point in time audit)
<= /li>InstrumentationThread.startThread(GrouperContext.retrie= veDefaultContext().getGrouperEngine(), customTypes);
Maybe there needs to be a way to add more types after s=
tartup so that the PSP (or something else) can add to it?
-- DOES= THIS WORK???? loaderRunOneJob("OTHER_JOB_tierInstrumentationDaemon");