[p2pu-dev] PD/Course Assessment/Metrics Mockups

Sun Jul 31 13:41:55 UTC 2011

Hi Jessica,

you might want to have a look at a recent presentation I gave, where I
talked about the possibility of accessing external contributions and
interesting ways of both providing metrics and visualizing these for
research and for the participants - this is very early stuff, and some of it
is quite hard, but I might be interested in working on some of this for my
PhD thesis. Anyway we should definitively coordinate our work. I'd love to
discuss this further with you at some point.

http://www.slideshare.net/houshuang/open-courses-and-informal-learning-in-a-web-20-world-a-research-agenda

In the meantime, the stuff you listed all looks very useful/interesting!
Stian

On Sun, Jul 31, 2011 at 10:03, Jessica Ledbetter <
jessica at jessicaledbetter.com> wrote:

> Greetings all (and especially Karen):
>
> Attached are some mockups of some course metrics' graphs/charts. I'd love
> to get feedback from everyone by Wednesday. If that's not enough time, let
> me know.  (Karen asked about getting a way to assess courses into P2PU and
> these are the first draft attempts at what it could look like.)
>
> We start by going to the course's admin page. Right now it's a button
> called "Edit Study Group" (or "Edit Course" or .. etc.) but I propose that
> that becomes "Administer Study Group" or "Study Group Admin" to better
> encompass the addition of metrics (and maybe some other stuff that's in my
> ideabox). Also, we'd have a link to them on the study group's sidebar in the
> box where Updates, Organizers, and Participant count are.
>
> Not obvious by this screenshot is that I also widened the box that this is
> in so that it's the same width as the P2PU header. More real estate for all
> this will be a very good thing :) In my experimentation, I wanted even more
> room!
>
> The first chart on the metric page is* "Overall Activity"*:
>
> [image: projectAdmin.png]
>
> *1:* Can download this graph as a graphic. This will come in handy when
> the graphs get too big for the real estate available or for inclusion in
> presentations/reports.
>
> *2: *When mousing over points on the chart, hint text pops up with more
> information. The line in this is an average of the 3 participants.
>
> *3: *Specifically for this chart, I was trying to show the "health" of the
> course. Missing are Task Edits, Task Creations, Page Views by Participants,
> Page Views by Non-Participants (this would have to be somewhere else). I can
> add those here too, of course. I'm not sure if Time on Page fits well with
> these other data points but I put it here since it was requested. Maybe
> another chart that is more web analytics like the page views and time could
> be together? I have a thought on "health" that might be better in its own
> email. However, I talk about it below after all the mockups ("Health
> Formula") :)
>
> *Overall:* This chart, and others, could become difficult to use with a
> lot of students in the course. But I can stretch it out so that you can
> scroll over to see it all or have a sort of zoom in feature so you can see
> the overall lines and zoom in on certain areas.
>
> Also, I can make it so that when you click a bar that you "drill down" to
> see more information like *when *the activity occurred for the person.
>
> *Next is "Activity Over Time"* which is a more specific metric for health
> reporting.
>
> [image: chartActivityOverTime.png]
>
> Activities listed right now are: comments, task edits, participants' last
> log in time, and how many tasks were added. We can add page views in phase
> 2. We could also have a running "total" showing as well. Requested was the
> ability to see by day, week, and total. Also, I can give this as a table.
>
> Next is* "Participant Comments."*
>
> [image: chartParticipantComments.png]
>
> Here we have a "total" for the day plus what each person did to contribute
> to this total. There would be a chart like this for all activity types so
> that you can see per day and total. This one is from the view of a
> non-organizer. The participants' names are not shown but the logged in user
> will be able to see his or her own place compared to others. If the viewer
> is an organizer, then he or she can see all names.
> *
> Question: *Should the organizers be listed in the charts? I think they
> should but just wanted to double check. You can also click a name and toggle
> off whether or not the person's data is shown on the chart.
>
> *Question:* Should the public be able to see these? Their view would not
> show participants' names, of course.
>
> *Roadmap:*
>
>    - Get feedback :) Update based on that.
>
>
>    - Phase 1 would be:
>
>
>    - Comments
>    - Task edits
>    - Task creations
>    - Privacy policy update by John
>
>
>    - Phase 2 would be:
>
>
>    - Page views
>    - Time on page
>    - Last login
>
>
> *Health Formula:*
>
> As I mentioned above, I have an idea about the "health" measurement. What
> if we came up with a formula that helps us see at a glance how "healthy" a
> course is?
>
> What about putting more weight on comments and logged in user page views vs
> non logged in user page views. Task creation is important but a
> workshop-style class could have 3 tasks vs a more traditional course could
> have many. So this would probably be more focused on a course/study group vs
> a dashboard of all courses.
>
> *Perhaps:*
>
> Health = Number of Task Comments * 20  + Number of Wall Comments * 10 +
> Number of Tasks Created * 5 + Number of Page Views by Logged In Users * 5 +
> Number of Page Views by Non Logged In Users + Task Edits + Amount of Time in
> Seconds Logged In Users Spent on Page * 3 [This could be very faulty though]
> + Amount of Time in Seconds Non Participants Spent on Page [Still faulty but
> would be great if it did work :) ]
>
> Once we have badges in place, we could add in how many "complete a
> task"/"get a badge."
>
> We could also somehow leave it up to the organizer what the "health
> formula."
>
> *And then:
>
> *This was longer than I meant but hopefully this shows what I'm thinking
> and makes sure I'm covering everything :) Please let me know if you have any
> questions!
>
> I'm looking forward to the feedback and really looking forward to coding
> this up! I think this will benefit many organizers.
>
> By the way, the chart maker is http://www.highcharts.com. It's free for
> non-profits and I think it fits in with our license. If not, I'll need to
> look for something else.
>
> Thank you!
> Jessica
>
> --
> Jessica Ledbetter
> http://jessicaledbetter.com
>
>
>
> _______________________________________________
> p2pu-dev mailing list
> p2pu-dev at lists.p2pu.org
> http://lists.p2pu.org/mailman/listinfo/p2pu-dev
>
>

-- 
http://reganmian.net/blog -- Random Stuff that Matters
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.p2pu.org/pipermail/p2pu-dev/attachments/20110731/ee85ffb9/attachment.html>