Versions Compared

Old Version 10

changes.mady.by.user Hyzer

Saved on

compared with

New Version 11

changes.mady.by.user Shilen Patel (duke.edu)

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Code Block
otherJob.tierInstrumentationDaemon.class = edu.internet2.middleware.grouper.instrumentation.TierInstrumentationDaemon
otherJob.tierInstrumentationDaemon.quartzCron = 0 0 2 * * ?

 

Collecting UI Counts (under development)

  • Data will be kept in the folder etc:attribute:instrumentationData
  • Collect counts of servlet requests, group adds/deletes, membership adds/deletes, folders adds/deletes
  • UI can start a new thread when the servlet first initializes
  • The new thread will enable stat collection (i.e. set some static variable)
  • Grouper api and ui code will update various static lists of timestamps indicating when each operation is done
  • Config option will see how often the thread will go through the timestamps and update the grouper database.  Lower means less gaps when the process is killed.
  • Another config option will specify the increments to keep counts of.  E.g. if we're keeping counts by 10 minutes or hour or day.
  • When the UI thread starts up, check to see if an "instrumentation.dat" file exists in the logs directory.  This file will contain the uuid of this instance.
  • If it doesn't exist, create it and create a corresponding attribute in grouper, e.g. etc:attribute:instrumentationData:instances:theuuid (def = etc:attribute:instrumentationData:instancesDef)
  • The instrumentation.dat file should have a trivial update whenever the thread flushes to the database just in case the system is cleaning old files.
  • There will be a group used for assignments - etc:attribute:instrumentationData:instrumentationDataGroup.
  • There will be a single assign multi valued attribute - etc:attribute:instrumentationData:instrumentationDataCounts (def = etc:attribute:instrumentationData:instrumentationDataCountsDef)
  • So etc:attribute:instrumentationData:instances:theuuid will be assigned to etc:attribute:instrumentationData:instrumentationDataGroup.  And on that assignment will live assignments with actual data (instrumentationDataCounts)

  • The value of the assignment on the assignment will be like:  

    Wiki Markup
    {"startTimestamp" : "1486753200000", "GROUP_ADD", "5", "GROUP_DELETE", "3"}

  • There may be multiple values added each time it runs.  For example, if the database is updated every hour and the increment is every 10 minutes, then it could add 6 of these.

    Wiki Markup
    {"startTimestamp" : "1486753200000", "GROUP_ADD", "5", "GROUP_DELETE", "3"}
{"startTimestamp" : "1486753800000", "GROUP_ADD", "2", "GROUP_DELETE", "6"}
{"startTimestamp" : "1486754400000", "GROUP_ADD", "1", "GROUP_DELETE", "2"}etc
  • The TIER instrumentation daemon will sends these to TIER and delete the values from Grouper.
  • The values won't be audited (user audit or point in time audit)
  • Code should be reusable for WS, loader, etc.

Notes

  • Keith is interested in LogStash
  • Scott is interested in Metrics (java library)