[p2pu-dev] PD/Course Assessment/Metrics Mockups

Sun Jul 31 16:04:18 UTC 2011

Thank you for the presentation link. I had not seen it and I was nodding
along :)

Pulling in participation on different platforms could be very interesting.
It wasn't part of the original requirements document but I think it'd be
very valuable too. As you said, there's also the challenge of linking up the
different accounts too so that the system knows that that rss feed is *your*
blog. In a way, we kind of have that with external links. We'd need to know
that the posts are specific to the course though (could do with category/tag
like how some planets subscribe to a certain tag on a person's blog). And
with github, it would be the specific repository that goes with that
course's "homework." Yes. This would be really interesting. I definitely
would like to explore that :) Might not be phase 1 but would help show how
active a course is across the platforms people may be using.

How should we coordinate? Via this list, a group on P2PU, p2pu-researchers
list, or something else?

Also, as you may know, this "PD/Course Assessment/Metrics" task is one of
the things I'm working on as new summer part-time assessment developer. But
it's not like I'm disappearing at the end of summer and can always transfer
knowledge to whomever! :)

On Sun, Jul 31, 2011 at 9:41 AM, Stian Håklev <shaklev at gmail.com> wrote:

> Hi Jessica,
>
> you might want to have a look at a recent presentation I gave, where I
> talked about the possibility of accessing external contributions and
> interesting ways of both providing metrics and visualizing these for
> research and for the participants - this is very early stuff, and some of it
> is quite hard, but I might be interested in working on some of this for my
> PhD thesis. Anyway we should definitively coordinate our work. I'd love to
> discuss this further with you at some point.
>
>
> http://www.slideshare.net/houshuang/open-courses-and-informal-learning-in-a-web-20-world-a-research-agenda
>
> In the meantime, the stuff you listed all looks very useful/interesting!
> Stian
>
> On Sun, Jul 31, 2011 at 10:03, Jessica Ledbetter <
> jessica at jessicaledbetter.com> wrote:
>
>> Greetings all (and especially Karen):
>>
>> Attached are some mockups of some course metrics' graphs/charts. I'd love
>> to get feedback from everyone by Wednesday. If that's not enough time, let
>> me know.  (Karen asked about getting a way to assess courses into P2PU and
>> these are the first draft attempts at what it could look like.)
>>
>> We start by going to the course's admin page. Right now it's a button
>> called "Edit Study Group" (or "Edit Course" or .. etc.) but I propose that
>> that becomes "Administer Study Group" or "Study Group Admin" to better
>> encompass the addition of metrics (and maybe some other stuff that's in my
>> ideabox). Also, we'd have a link to them on the study group's sidebar in the
>> box where Updates, Organizers, and Participant count are.
>>
>> Not obvious by this screenshot is that I also widened the box that this is
>> in so that it's the same width as the P2PU header. More real estate for all
>> this will be a very good thing :) In my experimentation, I wanted even more
>> room!
>>
>> The first chart on the metric page is* "Overall Activity"*:
>>
>> [image: projectAdmin.png]
>>
>> *1:* Can download this graph as a graphic. This will come in handy when
>> the graphs get too big for the real estate available or for inclusion in
>> presentations/reports.
>>
>> *2: *When mousing over points on the chart, hint text pops up with more
>> information. The line in this is an average of the 3 participants.
>>
>> *3: *Specifically for this chart, I was trying to show the "health" of
>> the course. Missing are Task Edits, Task Creations, Page Views by
>> Participants, Page Views by Non-Participants (this would have to be
>> somewhere else). I can add those here too, of course. I'm not sure if Time
>> on Page fits well with these other data points but I put it here since it
>> was requested. Maybe another chart that is more web analytics like the page
>> views and time could be together? I have a thought on "health" that might be
>> better in its own email. However, I talk about it below after all the
>> mockups ("Health Formula") :)
>>
>> *Overall:* This chart, and others, could become difficult to use with a
>> lot of students in the course. But I can stretch it out so that you can
>> scroll over to see it all or have a sort of zoom in feature so you can see
>> the overall lines and zoom in on certain areas.
>>
>> Also, I can make it so that when you click a bar that you "drill down" to
>> see more information like *when *the activity occurred for the person.
>>
>> *Next is "Activity Over Time"* which is a more specific metric for health
>> reporting.
>>
>> [image: chartActivityOverTime.png]
>>
>> Activities listed right now are: comments, task edits, participants' last
>> log in time, and how many tasks were added. We can add page views in phase
>> 2. We could also have a running "total" showing as well. Requested was the
>> ability to see by day, week, and total. Also, I can give this as a table.
>>
>> Next is* "Participant Comments."*
>>
>> [image: chartParticipantComments.png]
>>
>> Here we have a "total" for the day plus what each person did to contribute
>> to this total. There would be a chart like this for all activity types so
>> that you can see per day and total. This one is from the view of a
>> non-organizer. The participants' names are not shown but the logged in user
>> will be able to see his or her own place compared to others. If the viewer
>> is an organizer, then he or she can see all names.
>> *
>> Question: *Should the organizers be listed in the charts? I think they
>> should but just wanted to double check. You can also click a name and toggle
>> off whether or not the person's data is shown on the chart.
>>
>> *Question:* Should the public be able to see these? Their view would not
>> show participants' names, of course.
>>
>> *Roadmap:*
>>
>>    - Get feedback :) Update based on that.
>>
>>
>>    - Phase 1 would be:
>>
>>
>>    - Comments
>>    - Task edits
>>    - Task creations
>>    - Privacy policy update by John
>>
>>
>>    - Phase 2 would be:
>>
>>
>>    - Page views
>>    - Time on page
>>    - Last login
>>
>>
>> *Health Formula:*
>>
>> As I mentioned above, I have an idea about the "health" measurement. What
>> if we came up with a formula that helps us see at a glance how "healthy" a
>> course is?
>>
>> What about putting more weight on comments and logged in user page views
>> vs non logged in user page views. Task creation is important but a
>> workshop-style class could have 3 tasks vs a more traditional course could
>> have many. So this would probably be more focused on a course/study group vs
>> a dashboard of all courses.
>>
>> *Perhaps:*
>>
>> Health = Number of Task Comments * 20  + Number of Wall Comments * 10 +
>> Number of Tasks Created * 5 + Number of Page Views by Logged In Users * 5 +
>> Number of Page Views by Non Logged In Users + Task Edits + Amount of Time in
>> Seconds Logged In Users Spent on Page * 3 [This could be very faulty though]
>> + Amount of Time in Seconds Non Participants Spent on Page [Still faulty but
>> would be great if it did work :) ]
>>
>> Once we have badges in place, we could add in how many "complete a
>> task"/"get a badge."
>>
>> We could also somehow leave it up to the organizer what the "health
>> formula."
>>
>> *And then:
>>
>> *This was longer than I meant but hopefully this shows what I'm thinking
>> and makes sure I'm covering everything :) Please let me know if you have any
>> questions!
>>
>> I'm looking forward to the feedback and really looking forward to coding
>> this up! I think this will benefit many organizers.
>>
>> By the way, the chart maker is http://www.highcharts.com. It's free for
>> non-profits and I think it fits in with our license. If not, I'll need to
>> look for something else.
>>
>> Thank you!
>> Jessica
>>
>> --
>> Jessica Ledbetter
>> http://jessicaledbetter.com
>>
>>
>>
>> _______________________________________________
>> p2pu-dev mailing list
>> p2pu-dev at lists.p2pu.org
>> http://lists.p2pu.org/mailman/listinfo/p2pu-dev
>>
>>
>
>
> --
> http://reganmian.net/blog -- Random Stuff that Matters
>
>
> _______________________________________________
> p2pu-dev mailing list
> p2pu-dev at lists.p2pu.org
> http://lists.p2pu.org/mailman/listinfo/p2pu-dev
>
>

-- 
Jessica Ledbetter
http://jessicaledbetter.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.p2pu.org/pipermail/p2pu-dev/attachments/20110731/b9631661/attachment-0001.html>