Launch reports as indicator of social engagement

[jon-betts]

This is very WIP at the moment, but there is a desire to measure some forms of the level social engagement in an organization / course / assignment.

The initial suggestion was:

• For the first 5 annotators, how many times do they return to the assignment
• This was refined to: For the first 5 logins, how many times do they return to the assignment
  • Because we don't need annotators necessarily, just active users
  • This is easier because the type of data is the same (launches and launches, rather than annotations and launches)
• This was refined to: As a graph, show the number of launches per user, ordered by first launch date
  • This is better, because it shows all users

We might want to modify this:

• Do we need / want this per assignment? Is course or organization better?
• Do we want to exclude teachers or be able to toggle it?

Some alternative ideas

Launches / users - a quick approximation

If we take the time / unique user elements out of the above, then it boils down to average launches per user (per object). We already have the data to calculate this for various things and so we could present this number very easily.

Showing reply graphs

Much more ambitious, but might showing a graph (as in nodes and edges) of annotations and replies might be informative, or might not. It might inform which other types of graph metric would be useful. For example are replies very deep, or shallow? Is the pattern of replies spikey (some deep, some not) or relatively flat?

Brain storming other ideas for engagement

Should we think about what might indicate engagement more broadly, or go with this single idea?

Is there something there about engagement per user? Are some users highly "engaged" and some not at all?

[darylhedley]

Ok - so after reading this write up and having a few other catch ups, I think a good first approach to getting the data we need for the project is:

1. Have a global stat that is based upon all users and should live on the business dashboard (or organisations overview). This stat should be launches / user / assignment. So, how many launches per user within an assignment. This will help us know if we are making improvements to social engagement across the board. I wouldn't mind this being over a course rather than an assignment.

2. Have a metric based upon individual assignments. We currently don't have a good place to put this. But I would like to have a view of an assignment and the launches per user in a graph like described here:
   This was refined to: As a graph, show the number of launches per user, ordered by first launch date
   This is better, because it shows all users
   This will help us understand which assignments are doing better for us to dig into the data further if we need to.

I'm going to put it out there that since we discussed the idea of a reply graph - I cannot get it out of my head. I think this is a great idea Jon. I could imagine us using this in Product to find out where conversation is happening, how long threads are and who is keeping the conversation going - is it returning teachers who are proactively pushing the conversation or is it all student led?