Page History

...

The goal of this project is to centrally collect TIER data about Grouper deployment to help improve Grouper and give information to TIER constituents about Grouper usage. As of Grouper 2.3 Patch 62, no information is automatically sent to TIER as we are waiting for TIER to have a production collector. However, the Grouper UI has a new feature that allows local administrators to view some of the data that would be sent to TIER.

Overall Design

Grouper has a central database which can store information for a Grouper env at an institution
Each JVM process (API, WS, UI, Daemon, GSH, etccurrently daemon and UI only, more later) can periodically check in to the DB (e.g. every 6 hours1 hour)
- Let it know its UUID, type of process, number of tx of various types since last checkin, version, patch level, uptimeetc
Daily a new TIER instrumentation daemon could collate information in the database, and glean other information (e.g. is PSP or PSPNG running) and after consulting the discovery service, send a report to the TIER collector. This is currently not automatically enabled as we are waiting for a production TIER collector.

Discovery service

Simple static HTTP resource(s) that would be always/forever available (e.g. hosted at AWS?) to designate where the TIER collector(s) are
Gliffy Diagram
name grouperInstrumentation
Note, we can start out with one collector. Client failover is optional, the first endpoint might be used only for simple clients
e.g. request: GET (current)
- https://id.internet2.edu/ti/jrd/collector
- results: http://cerif.org/elk /
e.g. request: GET (outdated)
- https://s3.amazonaws.com/edu.internet2.tierinstrumentationcollector.discovery0/tierInstrumentationCollector_discovery.json
- https://s3-us-west-1.amazonaws.com/edu.internet2.tierinstrumentationcollector.discovery1/tierInstrumentationCollector_discovery.json

...

Simple REST endpoint that takes any name/value pairs in JSON in a simple structure of single valued strings
The collector can just store each resource it gets and doesnt care what the attributes are, so the components can change their data as they need
Of course the reporting and processing needs to take the attributes and values into account
e.g. submission: POST https://tiercollector1.internet2.edu/v1/collector/dailyReport

Code Block

{
   "reportFormat": 1,
   "component": "grouper",
  institution: "Penn"institution”:”Penn”,
  environment: "prod"environment”:”prod”,
  version: "version":"2.3.0",
  patchesInstalled: "platformWindows"api1:false,
 api2, api4, ws2, ws3" "platformLinux":false,
   wsServerCount: 3"platformMac":true,
   platformLinux: true"platformSolaris":false,
   uiServerCount: 1"transactionCountMemberships":6,
   pspngCount: 1"transactionCountPrivileges":0,
   provisionToLdap: true"registryCountDirectMemberships":2,
  registrySize: 12345678,
  transactionCountMemberships: 12432,
  transactionCountPrivileges: 432,
  transactionCountPermissions: 17
}

Schema on mysql (record table and attribute table)

Note: diagnostics should take into account generic daemon configs

...

 "registryCountDirectPrivileges":54,
   "registryCountDirectPermissions":0,
   "provisionToLdapUsingPsp":false,
   "provisionToLdapUsingPspng":false,
   "patchesInstalled":[  

   ],
   "instances":[  
      {  
         "uuid":"d0a363444b49489591b80690a6edc09d",
         "engineName":"grouperShell",
         "serverLabel":"-2",
         "lastUpdate":1489762214975,
         "newCounts":[  

         ]
      },
      {  
         "uuid":"2898de277109417ebccd00cee00f0649",
         "engineName":"grouperLoader",
         "serverLabel":"-2",
         "lastUpdate":1489769384218,
         "newCounts":[  

         ]
      },
      {  
         "uuid":"1a8885db0b564e94b3958dbfc7b032f0",
         "engineName":"grouperUI",
         "serverLabel":"-2",
         "lastUpdate":1489769384725,
         "newCounts":[  
            {  
               "startTime":1489767550000,
               "duration":5000,
               "UI_REQUESTS":5
            },
            {  
               "startTime":1489767600000,
               "duration":5000,
               "UI_REQUESTS":1
            },
            {  
               "startTime":1489767635000,
               "duration":5000,
               "UI_REQUESTS":1
            },
            {  
               "startTime":1489767650000,
               "duration":5000,
               "UI_REQUESTS":1
            },
            {  
               "startTime":1489767660000,
               "duration":5000,
               "API_GROUP_ADD":1,
               "UI_REQUESTS":2
            },
            {  
               "startTime":1489767690000,
               "duration":5000,
               "API_GROUP_DELETE":1,
               "UI_REQUESTS":3
            },
            {  
               "startTime":1489767720000,
               "duration":5000,
               "UI_REQUESTS":1
            },
            {  
               "startTime":1489767725000,
               "duration":5000,
               "UI_REQUESTS":3
            },
            {  
               "startTime":1489767730000,
               "duration":5000,
               "UI_REQUESTS":3,
               "API_STEM_ADD":1
            },
            {  
               "startTime":1489767735000,
               "duration":5000,
               "UI_REQUESTS":2,
               "API_STEM_DELETE":1
            },
            {  
               "startTime":1489767770000,
               "duration":5000,
               "UI_REQUESTS":1
            },
            {  
               "startTime":1489767790000,
               "duration":5000,
               "UI_REQUESTS":2
            },
            {  
               "startTime":1489767795000,
               "duration":5000,
               "UI_REQUESTS":2,
               "API_GROUP_ADD":1
            },
            {  
               "startTime":1489767800000,
               "duration":5000,
               "UI_REQUESTS":1
            },
            {  
               "startTime":1489767805000,
               "duration":5000,
               "API_MEMBERSHIP_ADD":1,
               "UI_REQUESTS":3
            },
            {  
               "startTime":1489767835000,
               "duration":5000,
               "UI_REQUESTS":1
            },
            {  
               "startTime":1489769030000,
               "duration":5000,
               "UI_REQUESTS":4
            },
            {  
               "startTime":1489769035000,
               "duration":5000,
               "UI_REQUESTS":1
            },
            {  
               "startTime":1489769045000,
               "duration":5000,
               "UI_REQUESTS":4
            },
            {  
               "startTime":1489769170000,
               "duration":5000,
               "UI_REQUESTS":1
            },
            {  
               "startTime":1489769175000,
               "duration":5000,
               "UI_REQUESTS":2
            },
            {  
               "startTime":1489769275000,
               "duration":5000,
               "API_MEMBERSHIP_DELETE":1,
               "UI_REQUESTS":1
            },
            {  
               "startTime":1489769315000,
               "duration":5000,
               "API_MEMBERSHIP_ADD":1,
               "UI_REQUESTS":4
            }
         ]
      }
   ]
}

Enable collection

Note this will go to a test collector. Shouldn't be used in production at the moment.

Get patches for 2.3 (24 and 25)

...

Page tree

Versions Compared

Old Version 29

New Version 30

Key

Overall Design

Discovery service

Enable collection