[p2pu-dev] Two way communication for notifications and messaging

Mon Jun 4 21:50:28 UTC 2012

So guys, SendGrid also has a parse API:
http://docs.sendgrid.com/documentation/api/parse-api-2/
and also an event API, and a Django app:
https://github.com/RyanBalfanz/django-sendgrid

cheers,
José


On 4 June 2012 22:17, Stian Håklev <shaklev at gmail.com> wrote:
> If not, there might be many who signed up to FB with one address (say their
> university address) which is now permanently forwarded to another e-mail
> (say a GMail account) who would not be able to respond, so maybe they just
> rely on the token itself.
>
> Anyway sender e-mail is very easy to spoof so checking that might not add
> much real security, while frustrating many users.
>
> Stian
>
>
> On Mon, Jun 4, 2012 at 5:05 PM, Jessy Kate Schingler <jessy at jessykate.com>
> wrote:
>>
>> i guess the hash in the email address would function as a kind of
>> authentication token, probably combined with the sender email address which
>> would have to match an email associated with the account.
>>
>> worth trying: someone try forwarding a FB notification email to a friend,
>> and see if that friend is able to reply-to-post with a different email using
>> :).
>>
>> On Mon, Jun 4, 2012 at 1:58 PM, Jos Flores <josmasflores at gmail.com> wrote:
>>>
>>> Stian, I understand the workflow but my question is: where do
>>> cloudmailin come in in that workflow? as far as I know mail now is
>>> handled by sendgrid; does that have to change? who creates the token?
>>> cloudmailin? lernanta in the codebase?
>>> The example they give is that it is just like a POST in a form in your
>>> site: in lernanta to do a POST in a reply form you have to be
>>> authenticated, so it's not a straight forward, open POST request.
>>>
>>> cheers,
>>> José
>>>
>>> On 4 June 2012 21:17, Stian Håklev <shaklev at gmail.com> wrote:
>>> > Well, I guess every single message is sent out with a unique token,
>>> > like
>>> >
>>> > reply-34093049304 at p2pu.org
>>> >
>>> > the combination of this token, and the sender e-mail address should be
>>> > enough authentication?
>>> >
>>> > Or do you mean how do we know that the HTTP posts from cloudmailin
>>> > actually
>>> > come from cloudmailin?
>>> >
>>> > Apart from the possible vicissitudes of handling email ourselves (which
>>> > I
>>> > have no experience with, but it always seems more complex than what
>>> > you'd
>>> > think - although perhaps that's more for sending email than for
>>> > receiving),
>>> > it really doesn't seem all that difficult to me.
>>> >
>>> > For each notification e-mail, generate a unique ID, and store that ID
>>> > and
>>> > the e-mail addresses that received the notification in a database. When
>>> > you
>>> > receive a reply, look up the ID, check that e-mail match, and look up
>>> > where
>>> > to append the reply.
>>> >
>>> > Stian
>>> >
>>> >
>>> > On Mon, Jun 4, 2012 at 4:13 PM, Jos Flores <josmasflores at gmail.com>
>>> > wrote:
>>> >>
>>> >> How does this handle authorisation though? I see they do have docs but
>>> >> I'm not following... Would it be related to the email account the user
>>> >> signed up on the site with?
>>> >>
>>> >> http://docs.cloudmailin.com/features/callbacks/#authorization-callbacks
>>> >>
>>> >> They even have a Django app all ready:
>>> >> https://github.com/CloudMailin/django-cloudmailin      ...tempting
>>> >> indeed! :)
>>> >>
>>> >> cheers,
>>> >> José
>>> >>
>>> >>
>>> >> On 4 June 2012 18:51, Jessy Kate Schingler <jessy at jessykate.com>
>>> >> wrote:
>>> >> > that's cool dirk i've not heard of cloudmailin!
>>> >> >
>>> >> > looking at their site it seems like a pretty neat service in
>>> >> > general,
>>> >> > but
>>> >> > i;m not sure how much it would buy us in terms of reducing the work
>>> >> > we
>>> >> > have
>>> >> > to do on the server side. doing it ourselves i guess we would write
>>> >> > a
>>> >> > script
>>> >> > that parses incoming mails in the postfix maildir or exim equivalent
>>> >> > (which
>>> >> > cloudmailin would take care of). but we still have to disambiguate
>>> >> > responses
>>> >> > into specific accounts/discussion threads/models.
>>> >> >
>>> >> > anyway, my sense is that it wouldn't save us a ton of effort, and
>>> >> > would
>>> >> > increase dependencies on external services, but curious what others
>>> >> > think.
>>> >> >
>>> >> > other thoughts?
>>> >> >
>>> >> >
>>> >> > On Mon, Jun 4, 2012 at 8:53 AM, Stian Håklev <shaklev at gmail.com>
>>> >> > wrote:
>>> >> >>
>>> >> >> To get a sense of the traffic levels, how many messages are posted
>>> >> >> every
>>> >> >> month on P2PU right now? (Of course, we'd then have to estimate
>>> >> >> roughly
>>> >> >> what
>>> >> >> percentage of those messages are likely to be posted through email
>>> >> >> rather
>>> >> >> than online).
>>> >> >>
>>> >> >> Stian
>>> >> >>
>>> >> >>
>>> >> >> On Mon, Jun 4, 2012 at 11:22 AM, Dirk Uys <dirk at p2pu.org> wrote:
>>> >> >>>
>>> >> >>> Googling around a bit I found http://www.cloudmailin.com/
>>> >> >>>
>>> >> >>> They turn your email into an http post to your website. I like the
>>> >> >>> idea, it keeps the email stuff separate from the messaging logic.
>>> >> >>>
>>> >> >>> They also have special pricing options for non-profits.
>>> >> >>>
>>> >> >>> The alternative would be to setup our own mail server and use some
>>> >> >>> python bindings for mail and a celery task. There's poplib and
>>> >> >>> imaplib
>>> >> >>> for python.
>>> >> >>>
>>> >> >>> Any thoughts
>>> >> >>>
>>> >> >>> Cheers
>>> >> >>> d
>>> >> >>>
>>> >> >>> On Sat, Jun 2, 2012 at 1:02 AM, Stian Håklev <shaklev at gmail.com>
>>> >> >>> wrote:
>>> >> >>> > A lot of emails disregard anything after a + in an email user
>>> >> >>> > name,
>>> >> >>> > like
>>> >> >>> > GMail which will send all mail to shaklev+test at gmail.com to
>>> >> >>> > shaklev at gmail.com. So if the server receives all mail to
>>> >> >>> > reply+dfsfd
>>> >> >>> > to
>>> >> >>> > the
>>> >> >>> > reply address, and the reply address is able to strip this out
>>> >> >>> > of
>>> >> >>> > the
>>> >> >>> > header, and then know where to put it, based on the unique ID,
>>> >> >>> > we
>>> >> >>> > should be
>>> >> >>> > good to go.
>>> >> >>> >
>>> >> >>> > I am sure there are services and APIs out there for doing this
>>> >> >>> > as
>>> >> >>> > well,
>>> >> >>> > but
>>> >> >>> > they are probably expensive for the kind of volume (although
>>> >> >>> > we'd
>>> >> >>> > have
>>> >> >>> > a lot
>>> >> >>> > less replies by email than we have email notifications).
>>> >> >>> >
>>> >> >>> > For such a common problem, you'd think there'd be a Django
>>> >> >>> > module or
>>> >> >>> > at
>>> >> >>> > least a Python implementation out there somewhere as well. But I
>>> >> >>> > don't
>>> >> >>> > know
>>> >> >>> > how much of the challenge is in the Lernanta codebase, and how
>>> >> >>> > much
>>> >> >>> > is
>>> >> >>> > the
>>> >> >>> > email server etc.
>>> >> >>> >
>>> >> >>> > Stian
>>> >> >>> >
>>> >> >>> >
>>> >> >>> > On Fri, Jun 1, 2012 at 1:44 PM, Jessy Kate Schingler
>>> >> >>> > <jessy at jessykate.com>
>>> >> >>> > wrote:
>>> >> >>> >>
>>> >> >>> >> really cool dirk.
>>> >> >>> >>
>>> >> >>> >> i dont have a particular mail server recommendation though have
>>> >> >>> >> personally
>>> >> >>> >> done more with postfix than exim.
>>> >> >>> >>
>>> >> >>> >> i just did a quick search for "comments reply by email" and
>>> >> >>> >> noticed
>>> >> >>> >> that
>>> >> >>> >> wordpress implemented this last year. obviously they're php but
>>> >> >>> >> it
>>> >> >>> >> might be
>>> >> >>> >> interested to take a look at the WP codebase to see how they
>>> >> >>> >> implemented it.
>>> >> >>> >>
>>> >> >>> >> looking at facebook notifications, they use emails structured
>>> >> >>> >> like
>>> >> >>> >> so:
>>> >> >>> >> update+ohlp1evf at facebookmail.com
>>> >> >>> >>
>>> >> >>> >> so i guess update@ is their updates email address and the hash
>>> >> >>> >> let's
>>> >> >>> >> them
>>> >> >>> >> disambiguate to a specific conversation thread via some kind of
>>> >> >>> >> script
>>> >> >>> >> internally. i think this would be a fun/interesting thing to
>>> >> >>> >> contribute to
>>> >> >>> >> if it gets going, but know my time will be somewhat limited
>>> >> >>> >> this
>>> >> >>> >> summer with
>>> >> >>> >> internship work.
>>> >> >>> >>
>>> >> >>> >> jessy
>>> >> >>> >>
>>> >> >>> >>
>>> >> >>> >> On Fri, Jun 1, 2012 at 6:26 AM, Dirk Uys <dirk at p2pu.org> wrote:
>>> >> >>> >>>
>>> >> >>> >>> Thanks, having a look at ack atm!
>>> >> >>> >>>
>>> >> >>> >>> On Fri, Jun 1, 2012 at 3:07 PM, Jos Flores
>>> >> >>> >>> <josmasflores at gmail.com>
>>> >> >>> >>> wrote:
>>> >> >>> >>> > Dirk, thanks for the walkthrough, it does make sense.
>>> >> >>> >>> >
>>> >> >>> >>> > I've noticed you recommend grep... have you tried ack? it's
>>> >> >>> >>> > really
>>> >> >>> >>> > cool!
>>> >> >>> >>> > I use grep very little since I discovered it :)
>>> >> >>> >>> >
>>> >> >>> >>> > cheers,
>>> >> >>> >>> > José
>>> >> >>> >>> >
>>> >> >>> >>> > On 1 June 2012 09:57, Dirk Uys <dirk at p2pu.org> wrote:
>>> >> >>> >>> >> No problem. At the moment the notification code isn't in
>>> >> >>> >>> >> one
>>> >> >>> >>> >> place
>>> >> >>> >>> >> (something we can work on).
>>> >> >>> >>> >>
>>> >> >>> >>> >> I'm going to start explaining from the point where the
>>> >> >>> >>> >> email is
>>> >> >>> >>> >> sent
>>> >> >>> >>> >> to the user and work my way backwards to where
>>> >> >>> >>> >> notifications
>>> >> >>> >>> >> are
>>> >> >>> >>> >> generated. Please tell me if this ends up being confusing
>>> >> >>> >>> >>
>>> >> >>> >>> >> Emails are sent to a user by calling
>>> >> >>> >>> >> django.contrib.auth.models.User.email_user
>>> >> >>> >>> >>
>>> >> >>> >>> >>
>>> >> >>> >>> >>
>>> >> >>> >>> >>
>>> >> >>> >>> >> (https://docs.djangoproject.com/en/dev/topics/auth/#django.contrib.auth.models.User.email_user)
>>> >> >>> >>> >>
>>> >> >>> >>> >> email_user() is called by an async celery task
>>> >> >>> >>> >> SendNotification
>>> >> >>> >>> >>
>>> >> >>> >>> >>
>>> >> >>> >>> >>
>>> >> >>> >>> >>
>>> >> >>> >>> >> (https://github.com/p2pu/lernanta/blob/master/lernanta/apps/users/tasks.py#L25)
>>> >> >>> >>> >>
>>> >> >>> >>> >> SendNotification tasks can be created in multiple places:
>>> >> >>> >>> >>
>>> >> >>> >>> >> There is a signal
>>> >> >>> >>> >>
>>> >> >>> >>> >>
>>> >> >>> >>> >>
>>> >> >>> >>> >>
>>> >> >>> >>> >> (https://github.com/p2pu/lernanta/blob/master/lernanta/apps/content/models.py#L245)
>>> >> >>> >>> >> that fires when tasks are created or modified. This signal
>>> >> >>> >>> >> then
>>> >> >>> >>> >> calls
>>> >> >>> >>> >> a function called send_email_notification
>>> >> >>> >>> >>
>>> >> >>> >>> >>
>>> >> >>> >>> >>
>>> >> >>> >>> >>
>>> >> >>> >>> >> (https://github.com/p2pu/lernanta/blob/master/lernanta/apps/content/models.py#L213)
>>> >> >>> >>> >> that creates SendNotifications tasks.
>>> >> >>> >>> >>
>>> >> >>> >>> >> Another example is sent_creation_notification
>>> >> >>> >>> >>
>>> >> >>> >>> >>
>>> >> >>> >>> >>
>>> >> >>> >>> >>
>>> >> >>> >>> >> (https://github.com/p2pu/lernanta/blob/master/lernanta/apps/projects/models.py#L270)
>>> >> >>> >>> >> thats part of projects.models.Project. This function is
>>> >> >>> >>> >> called
>>> >> >>> >>> >> from
>>> >> >>> >>> >> the create function
>>> >> >>> >>> >>
>>> >> >>> >>> >>
>>> >> >>> >>> >>
>>> >> >>> >>> >>
>>> >> >>> >>> >> (https://github.com/p2pu/lernanta/blob/master/lernanta/apps/projects/models.py#L270/)
>>> >> >>> >>> >> that in turn gets called in the create view
>>> >> >>> >>> >>
>>> >> >>> >>> >>
>>> >> >>> >>> >>
>>> >> >>> >>> >>
>>> >> >>> >>> >> (https://github.com/p2pu/lernanta/blob/master/lernanta/apps/projects/views.py#L189)
>>> >> >>> >>> >>
>>> >> >>> >>> >> I'm not sure at what other places SendNotification tasks
>>> >> >>> >>> >> are
>>> >> >>> >>> >> created,
>>> >> >>> >>> >> but you can grep to code base to find out. grep -r
>>> >> >>> >>> >> SendNotification *
>>> >> >>> >>> >>
>>> >> >>> >>> >> The fact that the places where SendNotification tasks are
>>> >> >>> >>> >> created
>>> >> >>> >>> >> are
>>> >> >>> >>> >> spread all over isn't a problem in itself, but at the
>>> >> >>> >>> >> moment
>>> >> >>> >>> >> checks
>>> >> >>> >>> >> for notification preference also seem spread out a bit. For
>>> >> >>> >>> >> example
>>> >> >>> >>> >> task updates
>>> >> >>> >>> >>
>>> >> >>> >>> >>
>>> >> >>> >>> >>
>>> >> >>> >>> >> (https://github.com/p2pu/lernanta/blob/master/lernanta/apps/content/models.py#L234).
>>> >> >>> >>> >> It also makes it difficult to implement batching of
>>> >> >>> >>> >> notifications.
>>> >> >>> >>> >>
>>> >> >>> >>> >> Than being said, I think that notification may be one of
>>> >> >>> >>> >> the
>>> >> >>> >>> >> first
>>> >> >>> >>> >> parts of Lernanta that we can put into a separate module
>>> >> >>> >>> >> without
>>> >> >>> >>> >> too
>>> >> >>> >>> >> much surgery on the whole code base.
>>> >> >>> >>> >>
>>> >> >>> >>> >> Cheers
>>> >> >>> >>> >> d
>>> >> >>> >>> >>
>>> >> >>> >>> >> On Fri, Jun 1, 2012 at 10:28 AM, Jos Flores
>>> >> >>> >>> >> <josmasflores at gmail.com>
>>> >> >>> >>> >> wrote:
>>> >> >>> >>> >>> Hey Dirk,
>>> >> >>> >>> >>>
>>> >> >>> >>> >>> I know nothing about the topic but curious about where the
>>> >> >>> >>> >>> notifications code is and how it works... could you point
>>> >> >>> >>> >>> me
>>> >> >>> >>> >>> towards
>>> >> >>> >>> >>> it?
>>> >> >>> >>> >>>
>>> >> >>> >>> >>> cheers,
>>> >> >>> >>> >>> José
>>> >> >>> >>> >>>
>>> >> >>> >>> >>>
>>> >> >>> >>> >>> On 1 June 2012 09:01, Dirk Uys <dirk at p2pu.org> wrote:
>>> >> >>> >>> >>>> Hi
>>> >> >>> >>> >>>>
>>> >> >>> >>> >>>> I've been thinking a little bit about two way interaction
>>> >> >>> >>> >>>> for
>>> >> >>> >>> >>>> messaging and notifications. On the software side of
>>> >> >>> >>> >>>> things I
>>> >> >>> >>> >>>> don't
>>> >> >>> >>> >>>> foresee too much trouble, but on the config side of
>>> >> >>> >>> >>>> things
>>> >> >>> >>> >>>> I'm a
>>> >> >>> >>> >>>> little uncertain.
>>> >> >>> >>> >>>>
>>> >> >>> >>> >>>> Our primary MX record points to google, then
>>> >> >>> >>> >>>> @lists.p2pu.org
>>> >> >>> >>> >>>> points
>>> >> >>> >>> >>>> to
>>> >> >>> >>> >>>> the service we use for the mailing list. I guess we could
>>> >> >>> >>> >>>> use
>>> >> >>> >>> >>>> something like @site.p2pu.org? I see that github uses
>>> >> >>> >>> >>>> @reply.github.com with something that looks like a one
>>> >> >>> >>> >>>> time
>>> >> >>> >>> >>>> token.
>>> >> >>> >>> >>>>
>>> >> >>> >>> >>>> Is there any specific mail server that anyone would
>>> >> >>> >>> >>>> suggest?
>>> >> >>> >>> >>>> I
>>> >> >>> >>> >>>> have
>>> >> >>> >>> >>>> always skipped the talk about mail servers when reading
>>> >> >>> >>> >>>> linux
>>> >> >>> >>> >>>> sysadmin
>>> >> >>> >>> >>>> docs :)
>>> >> >>> >>> >>>>
>>> >> >>> >>> >>>> Cheers
>>> >> >>> >>> >>>> d
>>> >> >>> >>> >>>>
>>> >> >>> >>> >>>> ps. I apologize for the thinking as I type email
>>> >> >>> >>> >>>> _______________________________________________
>>> >> >>> >>> >>>> p2pu-dev mailing list
>>> >> >>> >>> >>>> p2pu-dev at lists.p2pu.org
>>> >> >>> >>> >>>> http://lists.p2pu.org/mailman/listinfo/p2pu-dev
>>> >> >>> >>> >>> _______________________________________________
>>> >> >>> >>> >>> p2pu-dev mailing list
>>> >> >>> >>> >>> p2pu-dev at lists.p2pu.org
>>> >> >>> >>> >>> http://lists.p2pu.org/mailman/listinfo/p2pu-dev
>>> >> >>> >>> >> _______________________________________________
>>> >> >>> >>> >> p2pu-dev mailing list
>>> >> >>> >>> >> p2pu-dev at lists.p2pu.org
>>> >> >>> >>> >> http://lists.p2pu.org/mailman/listinfo/p2pu-dev
>>> >> >>> >>> > _______________________________________________
>>> >> >>> >>> > p2pu-dev mailing list
>>> >> >>> >>> > p2pu-dev at lists.p2pu.org
>>> >> >>> >>> > http://lists.p2pu.org/mailman/listinfo/p2pu-dev
>>> >> >>> >>> _______________________________________________
>>> >> >>> >>> p2pu-dev mailing list
>>> >> >>> >>> p2pu-dev at lists.p2pu.org
>>> >> >>> >>> http://lists.p2pu.org/mailman/listinfo/p2pu-dev
>>> >> >>> >>
>>> >> >>> >>
>>> >> >>> >>
>>> >> >>> >>
>>> >> >>> >> --
>>> >> >>> >> Jessy
>>> >> >>> >> http://jessykate.com
>>> >> >>> >>
>>> >> >>> >>
>>> >> >>> >> _______________________________________________
>>> >> >>> >> p2pu-dev mailing list
>>> >> >>> >> p2pu-dev at lists.p2pu.org
>>> >> >>> >> http://lists.p2pu.org/mailman/listinfo/p2pu-dev
>>> >> >>> >>
>>> >> >>> >
>>> >> >>> >
>>> >> >>> >
>>> >> >>> > --
>>> >> >>> > http://reganmian.net/blog -- Random Stuff that Matters
>>> >> >>> >
>>> >> >>> >
>>> >> >>> > _______________________________________________
>>> >> >>> > p2pu-dev mailing list
>>> >> >>> > p2pu-dev at lists.p2pu.org
>>> >> >>> > http://lists.p2pu.org/mailman/listinfo/p2pu-dev
>>> >> >>> >
>>> >> >>> _______________________________________________
>>> >> >>> p2pu-dev mailing list
>>> >> >>> p2pu-dev at lists.p2pu.org
>>> >> >>> http://lists.p2pu.org/mailman/listinfo/p2pu-dev
>>> >> >>
>>> >> >>
>>> >> >>
>>> >> >>
>>> >> >> --
>>> >> >> http://reganmian.net/blog -- Random Stuff that Matters
>>> >> >>
>>> >> >>
>>> >> >> _______________________________________________
>>> >> >> p2pu-dev mailing list
>>> >> >> p2pu-dev at lists.p2pu.org
>>> >> >> http://lists.p2pu.org/mailman/listinfo/p2pu-dev
>>> >> >>
>>> >> >
>>> >> >
>>> >> >
>>> >> > --
>>> >> > Jessy
>>> >> > http://jessykate.com
>>> >> >
>>> >> >
>>> >> > _______________________________________________
>>> >> > p2pu-dev mailing list
>>> >> > p2pu-dev at lists.p2pu.org
>>> >> > http://lists.p2pu.org/mailman/listinfo/p2pu-dev
>>> >> >
>>> >> _______________________________________________
>>> >> p2pu-dev mailing list
>>> >> p2pu-dev at lists.p2pu.org
>>> >> http://lists.p2pu.org/mailman/listinfo/p2pu-dev
>>> >
>>> >
>>> >
>>> >
>>> > --
>>> > http://reganmian.net/blog -- Random Stuff that Matters
>>> >
>>> >
>>> > _______________________________________________
>>> > p2pu-dev mailing list
>>> > p2pu-dev at lists.p2pu.org
>>> > http://lists.p2pu.org/mailman/listinfo/p2pu-dev
>>> >
>>> _______________________________________________
>>> p2pu-dev mailing list
>>> p2pu-dev at lists.p2pu.org
>>> http://lists.p2pu.org/mailman/listinfo/p2pu-dev
>>
>>
>>
>>
>> --
>> Jessy
>> http://jessykate.com
>>
>>
>> _______________________________________________
>> p2pu-dev mailing list
>> p2pu-dev at lists.p2pu.org
>> http://lists.p2pu.org/mailman/listinfo/p2pu-dev
>>
>
>
>
> --
> http://reganmian.net/blog -- Random Stuff that Matters
>
>
> _______________________________________________
> p2pu-dev mailing list
> p2pu-dev at lists.p2pu.org
> http://lists.p2pu.org/mailman/listinfo/p2pu-dev
>