...
Experience at Grouper Demo
Pre-patch
| Code Block |
|---|
-- Audit entries: 137210 select count(1) from grouper_audit_entry_v; -- Audit entries no logged in user: 138204128204 select count(1) from grouper_audit_entry_v where logged_in_subject_id is null; -- Audit entries no logged in user older than a month old: 82201 select count(1) from grouper_audit_entry_v where logged_in_subject_id is null and created_on < 1516338000000; -- PIT entries: 148907 SELECT (SELECT COUNT(1) FROM grouper_pit_attribute_assign) + (SELECT COUNT(1) FROM grouper_pit_attribute_def) + (SELECT COUNT(1) FROM grouper_pit_attr_assn_actn) + (SELECT COUNT(1) FROM grouper_pit_attr_assn_actn_set) + (SELECT COUNT(1) FROM grouper_pit_attr_assn_value) + (SELECT COUNT(1) FROM grouper_pit_attr_def_name) + (SELECT COUNT(1) FROM grouper_pit_attr_def_name_set) + (SELECT COUNT(1) FROM grouper_pit_fields) + (SELECT COUNT(1) FROM grouper_pit_groups) + (SELECT COUNT(1) FROM grouper_pit_group_set) + (SELECT COUNT(1) FROM grouper_pit_members) + (SELECT COUNT(1) FROM grouper_pit_memberships) + (SELECT COUNT(1) FROM grouper_pit_role_set) + (SELECT COUNT(1) FROM grouper_pit_stems) AS pit_count FROM DUAL; -- PIT entries of deleted objects: 126095 SELECT (SELECT COUNT(1) FROM grouper_pit_attribute_assign WHERE end_time IS NOT NULL) + (SELECT COUNT(1) FROM grouper_pit_attribute_def WHERE end_time IS NOT NULL) + (SELECT COUNT(1) FROM grouper_pit_attr_assn_actn WHERE end_time IS NOT NULL) + (SELECT COUNT(1) FROM grouper_pit_attr_assn_actn_set WHERE end_time IS NOT NULL) + (SELECT COUNT(1) FROM grouper_pit_attr_assn_value WHERE end_time IS NOT NULL) + (SELECT COUNT(1) FROM grouper_pit_attr_def_name WHERE end_time IS NOT NULL) + (SELECT COUNT(1) FROM grouper_pit_attr_def_name_set WHERE end_time IS NOT NULL) + (SELECT COUNT(1) FROM grouper_pit_fields WHERE end_time IS NOT NULL) + (SELECT COUNT(1) FROM grouper_pit_groups WHERE end_time IS NOT NULL) + (SELECT COUNT(1) FROM grouper_pit_group_set WHERE end_time IS NOT NULL) + (SELECT COUNT(1) FROM grouper_pit_members WHERE end_time IS NOT NULL) + (SELECT COUNT(1) FROM grouper_pit_memberships WHERE end_time IS NOT NULL) + (SELECT COUNT(1) FROM grouper_pit_role_set WHERE end_time IS NOT NULL) + (SELECT COUNT(1) FROM grouper_pit_stems WHERE end_time IS NOT NULL) AS pit_count FROM DUAL; -- PIT entries of deleted objects older than a month: 80342 SELECT (SELECT COUNT(1) FROM grouper_pit_attribute_assign WHERE end_time IS NOT NULL and end_time < 1516338000000000) + (SELECT COUNT(1) FROM grouper_pit_attribute_def WHERE end_time IS NOT NULL and end_time < 1516338000000000) + (SELECT COUNT(1) FROM grouper_pit_attr_assn_actn WHERE end_time IS NOT NULL and end_time < 1516338000000000) + (SELECT COUNT(1) FROM grouper_pit_attr_assn_actn_set WHERE end_time IS NOT NULL and end_time < 1516338000000000) + (SELECT COUNT(1) FROM grouper_pit_attr_assn_value WHERE end_time IS NOT NULL and end_time < 1516338000000000) + (SELECT COUNT(1) FROM grouper_pit_attr_def_name WHERE end_time IS NOT NULL and end_time < 1516338000000000) + (SELECT COUNT(1) FROM grouper_pit_attr_def_name_set WHERE end_time IS NOT NULL and end_time < 1516338000000000) + (SELECT COUNT(1) FROM grouper_pit_fields WHERE end_time IS NOT NULL and end_time < 1516338000000000) + (SELECT COUNT(1) FROM grouper_pit_groups WHERE end_time IS NOT NULL and end_time < 1516338000000000) + (SELECT COUNT(1) FROM grouper_pit_group_set WHERE end_time IS NOT NULL and end_time < 1516338000000000) + (SELECT COUNT(1) FROM grouper_pit_members WHERE end_time IS NOT NULL and end_time < 1516338000000000) + (SELECT COUNT(1) FROM grouper_pit_memberships WHERE end_time IS NOT NULL and end_time < 1516338000000000) + (SELECT COUNT(1) FROM grouper_pit_role_set WHERE end_time IS NOT NULL and end_time < 1516338000000000) + (SELECT COUNT(1) FROM grouper_pit_stems WHERE end_time IS NOT NULL and end_time < 1516338000000000) AS pit_count FROM DUAL; -- Folders in delete folder 'users:penn' newer than 5 years: 5 SELECT COUNT(1) FROM grouper_stems WHERE NAME LIKE 'users:penn:%' AND create_time > 1361250000000; -- Folders in delete folder 'users:penn' older than 5 years: 39 SELECT COUNT(1) FROM grouper_stems WHERE NAME LIKE 'users:penn:%' AND create_time < 1361250000000; |
Post-patch: (dont configure anything)
Run
...
the
...
clean
...
logs
...
job
| Code Block |
|---|
gsh 0% grouperSession = GrouperSession.startRootSession(); edu.internet2.middleware.grouper.GrouperSession: 7f8474d2914a4cecac143be77768a34b,'GrouperSystem','application' gsh 1% loaderRunOneJob("MAINTENANCE_cleanLogs"); loader ran successfully: Deleted 3872 records from grouper_loader_log older than 7 days old. Deleted 2038 records from grouper_change_log_entry older than 14 days old. (1517867128940000) Deleted 0 instrumentation records older than 30 days old. (1516484734612) Configured to not delete records from audit_entry table with null logged in member idConfigured to not delete records from audit_entry tableConfigured to not delete records from DeletedPointInTimeObjects |
Do counts
| Code Block |
|---|
Audit entries: 137214
Audit entries no logged in user: 128207
Audit entries no logged in user older than a month old: 82201
PIT entries: 148913
PIT entries of deleted objects: 126096
PIT entries of deleted objects older than a month: 80342
Folders in delete folder 'users:penn' newer than 5 years: 5
Folders in delete folder 'users:penn' older than 5 years: 39 |
Configure deletes in grouper-loader.properties
| Code Block |
|---|
############################################
## audit entries with no logged in user aren't really all that useful. There is point in time data still.
## So removing these shouldn't be a big deal
## default is remove these that are 5 years old.
############################################
# number of days to retain db rows in grouper_audit_entry with no logged in user (loader, gsh, etc). -1 is forever.
# suggested is 365 or five years: 1825. Default is -1
loader.retain.db.audit_entry_no_logged_in_user.days=31
############################################
## After you delete an object in grouper, it is still in point in time. So if you want to know who
## was in a group a year ago, you need this info
## However, after some time it might be ok to let it go. So the default is 5 years
############################################
# number of days to retain db rows for point in time deleted objects. -1 is forever. suggested is 365 or five years: 1825.
# Default is -1
loader.retain.db.point_in_time_deleted_objects.days=31
############################################
## This is optional. You can just automatically obliterate folders *directly in a parent folder*
## that are a certain age old e.g. courses.
## so you could delete a term of courses 4 years old if you like. Note, make sure the loader isn't going to recreate
## or you will get churn
## Note this will also delete the point in time data as well.
############################################
# number of days after a subfolder (directly in a parent folder) is created that it will be obliterated (deleted)
# and point in time will be deleted too.
# "courses" or "anotherLabel" are variables you make up in these examples
loader.retain.db.folder.test.days=1825
loader.retain.db.folder.test.parentFolderName=users:penn
loader.retain.db.folder.test.deletePointInTime=true |
Run the daemon with GSH
| Code Block |
|---|
gsh 0% grouperSession = GrouperSession.startRootSession();
edu.internet2.middleware.grouper.GrouperSession: 7f8474d2914a4cecac143be77768a34b,'GrouperSystem','application'
gsh 1% loaderRunOneJob("MAINTENANCE_cleanLogs");
loader ran successfully: Deleted 3872 records from grouper_loader_log older than 7 days old. Deleted 2038 records from grouper_change_log_entry older than 14 days old. (1517867128940000) Deleted 0 instrumentation records older than 30 days old. (1516484734612) Configured to not delete records from audit_entry table with null logged in member idConfigured to not delete records from audit_entry tableConfigured to not delete records from DeletedPointInTimeObjects |
Counts afterwards
| Code Block |
|---|
Audit entries: 53540
Audit entries no logged in user: 44657
Audit entries no logged in user older than a month old: 0
PIT entries: 66930
PIT entries of deleted objects: 44508
PIT entries of deleted objects older than a month: 0
Folders in delete folder 'users:penn' newer than 5 years: 5
Folders in delete folder 'users:penn' older than 5 years: |
Experience at Penn test env
...