[p2pu-dev] Two way communication for notifications and messaging

Jos Flores josmasflores at gmail.com
Mon Jun 4 20:58:56 UTC 2012 

Stian, I understand the workflow but my question is: where do
cloudmailin come in in that workflow? as far as I know mail now is
handled by sendgrid; does that have to change? who creates the token?
cloudmailin? lernanta in the codebase?
The example they give is that it is just like a POST in a form in your
site: in lernanta to do a POST in a reply form you have to be
authenticated, so it's not a straight forward, open POST request.

cheers,
José

On 4 June 2012 21:17, Stian Håklev <shaklev at gmail.com> wrote:
> Well, I guess every single message is sent out with a unique token, like
>
> reply-34093049304 at p2pu.org
>
> the combination of this token, and the sender e-mail address should be
> enough authentication?
>
> Or do you mean how do we know that the HTTP posts from cloudmailin actually
> come from cloudmailin?
>
> Apart from the possible vicissitudes of handling email ourselves (which I
> have no experience with, but it always seems more complex than what you'd
> think - although perhaps that's more for sending email than for receiving),
> it really doesn't seem all that difficult to me.
>
> For each notification e-mail, generate a unique ID, and store that ID and
> the e-mail addresses that received the notification in a database. When you
> receive a reply, look up the ID, check that e-mail match, and look up where
> to append the reply.
>
> Stian
>
>
> On Mon, Jun 4, 2012 at 4:13 PM, Jos Flores <josmasflores at gmail.com> wrote:
>>
>> How does this handle authorisation though? I see they do have docs but
>> I'm not following... Would it be related to the email account the user
>> signed up on the site with?
>> http://docs.cloudmailin.com/features/callbacks/#authorization-callbacks
>>
>> They even have a Django app all ready:
>> https://github.com/CloudMailin/django-cloudmailin      ...tempting
>> indeed! :)
>>
>> cheers,
>> José
>>
>>
>> On 4 June 2012 18:51, Jessy Kate Schingler <jessy at jessykate.com> wrote:
>> > that's cool dirk i've not heard of cloudmailin!
>> >
>> > looking at their site it seems like a pretty neat service in general,
>> > but
>> > i;m not sure how much it would buy us in terms of reducing the work we
>> > have
>> > to do on the server side. doing it ourselves i guess we would write a
>> > script
>> > that parses incoming mails in the postfix maildir or exim equivalent
>> > (which
>> > cloudmailin would take care of). but we still have to disambiguate
>> > responses
>> > into specific accounts/discussion threads/models.
>> >
>> > anyway, my sense is that it wouldn't save us a ton of effort, and would
>> > increase dependencies on external services, but curious what others
>> > think.
>> >
>> > other thoughts?
>> >
>> >
>> > On Mon, Jun 4, 2012 at 8:53 AM, Stian Håklev <shaklev at gmail.com> wrote:
>> >>
>> >> To get a sense of the traffic levels, how many messages are posted
>> >> every
>> >> month on P2PU right now? (Of course, we'd then have to estimate roughly
>> >> what
>> >> percentage of those messages are likely to be posted through email
>> >> rather
>> >> than online).
>> >>
>> >> Stian
>> >>
>> >>
>> >> On Mon, Jun 4, 2012 at 11:22 AM, Dirk Uys <dirk at p2pu.org> wrote:
>> >>>
>> >>> Googling around a bit I found http://www.cloudmailin.com/
>> >>>
>> >>> They turn your email into an http post to your website. I like the
>> >>> idea, it keeps the email stuff separate from the messaging logic.
>> >>>
>> >>> They also have special pricing options for non-profits.
>> >>>
>> >>> The alternative would be to setup our own mail server and use some
>> >>> python bindings for mail and a celery task. There's poplib and imaplib
>> >>> for python.
>> >>>
>> >>> Any thoughts
>> >>>
>> >>> Cheers
>> >>> d
>> >>>
>> >>> On Sat, Jun 2, 2012 at 1:02 AM, Stian Håklev <shaklev at gmail.com>
>> >>> wrote:
>> >>> > A lot of emails disregard anything after a + in an email user name,
>> >>> > like
>> >>> > GMail which will send all mail to shaklev+test at gmail.com to
>> >>> > shaklev at gmail.com. So if the server receives all mail to reply+dfsfd
>> >>> > to
>> >>> > the
>> >>> > reply address, and the reply address is able to strip this out of
>> >>> > the
>> >>> > header, and then know where to put it, based on the unique ID, we
>> >>> > should be
>> >>> > good to go.
>> >>> >
>> >>> > I am sure there are services and APIs out there for doing this as
>> >>> > well,
>> >>> > but
>> >>> > they are probably expensive for the kind of volume (although we'd
>> >>> > have
>> >>> > a lot
>> >>> > less replies by email than we have email notifications).
>> >>> >
>> >>> > For such a common problem, you'd think there'd be a Django module or
>> >>> > at
>> >>> > least a Python implementation out there somewhere as well. But I
>> >>> > don't
>> >>> > know
>> >>> > how much of the challenge is in the Lernanta codebase, and how much
>> >>> > is
>> >>> > the
>> >>> > email server etc.
>> >>> >
>> >>> > Stian
>> >>> >
>> >>> >
>> >>> > On Fri, Jun 1, 2012 at 1:44 PM, Jessy Kate Schingler
>> >>> > <jessy at jessykate.com>
>> >>> > wrote:
>> >>> >>
>> >>> >> really cool dirk.
>> >>> >>
>> >>> >> i dont have a particular mail server recommendation though have
>> >>> >> personally
>> >>> >> done more with postfix than exim.
>> >>> >>
>> >>> >> i just did a quick search for "comments reply by email" and noticed
>> >>> >> that
>> >>> >> wordpress implemented this last year. obviously they're php but it
>> >>> >> might be
>> >>> >> interested to take a look at the WP codebase to see how they
>> >>> >> implemented it.
>> >>> >>
>> >>> >> looking at facebook notifications, they use emails structured like
>> >>> >> so:
>> >>> >> update+ohlp1evf at facebookmail.com
>> >>> >>
>> >>> >> so i guess update@ is their updates email address and the hash
>> >>> >> let's
>> >>> >> them
>> >>> >> disambiguate to a specific conversation thread via some kind of
>> >>> >> script
>> >>> >> internally. i think this would be a fun/interesting thing to
>> >>> >> contribute to
>> >>> >> if it gets going, but know my time will be somewhat limited this
>> >>> >> summer with
>> >>> >> internship work.
>> >>> >>
>> >>> >> jessy
>> >>> >>
>> >>> >>
>> >>> >> On Fri, Jun 1, 2012 at 6:26 AM, Dirk Uys <dirk at p2pu.org> wrote:
>> >>> >>>
>> >>> >>> Thanks, having a look at ack atm!
>> >>> >>>
>> >>> >>> On Fri, Jun 1, 2012 at 3:07 PM, Jos Flores
>> >>> >>> <josmasflores at gmail.com>
>> >>> >>> wrote:
>> >>> >>> > Dirk, thanks for the walkthrough, it does make sense.
>> >>> >>> >
>> >>> >>> > I've noticed you recommend grep... have you tried ack? it's
>> >>> >>> > really
>> >>> >>> > cool!
>> >>> >>> > I use grep very little since I discovered it :)
>> >>> >>> >
>> >>> >>> > cheers,
>> >>> >>> > José
>> >>> >>> >
>> >>> >>> > On 1 June 2012 09:57, Dirk Uys <dirk at p2pu.org> wrote:
>> >>> >>> >> No problem. At the moment the notification code isn't in one
>> >>> >>> >> place
>> >>> >>> >> (something we can work on).
>> >>> >>> >>
>> >>> >>> >> I'm going to start explaining from the point where the email is
>> >>> >>> >> sent
>> >>> >>> >> to the user and work my way backwards to where notifications
>> >>> >>> >> are
>> >>> >>> >> generated. Please tell me if this ends up being confusing
>> >>> >>> >>
>> >>> >>> >> Emails are sent to a user by calling
>> >>> >>> >> django.contrib.auth.models.User.email_user
>> >>> >>> >>
>> >>> >>> >>
>> >>> >>> >>
>> >>> >>> >> (https://docs.djangoproject.com/en/dev/topics/auth/#django.contrib.auth.models.User.email_user)
>> >>> >>> >>
>> >>> >>> >> email_user() is called by an async celery task SendNotification
>> >>> >>> >>
>> >>> >>> >>
>> >>> >>> >>
>> >>> >>> >> (https://github.com/p2pu/lernanta/blob/master/lernanta/apps/users/tasks.py#L25)
>> >>> >>> >>
>> >>> >>> >> SendNotification tasks can be created in multiple places:
>> >>> >>> >>
>> >>> >>> >> There is a signal
>> >>> >>> >>
>> >>> >>> >>
>> >>> >>> >>
>> >>> >>> >> (https://github.com/p2pu/lernanta/blob/master/lernanta/apps/content/models.py#L245)
>> >>> >>> >> that fires when tasks are created or modified. This signal then
>> >>> >>> >> calls
>> >>> >>> >> a function called send_email_notification
>> >>> >>> >>
>> >>> >>> >>
>> >>> >>> >>
>> >>> >>> >> (https://github.com/p2pu/lernanta/blob/master/lernanta/apps/content/models.py#L213)
>> >>> >>> >> that creates SendNotifications tasks.
>> >>> >>> >>
>> >>> >>> >> Another example is sent_creation_notification
>> >>> >>> >>
>> >>> >>> >>
>> >>> >>> >>
>> >>> >>> >> (https://github.com/p2pu/lernanta/blob/master/lernanta/apps/projects/models.py#L270)
>> >>> >>> >> thats part of projects.models.Project. This function is called
>> >>> >>> >> from
>> >>> >>> >> the create function
>> >>> >>> >>
>> >>> >>> >>
>> >>> >>> >>
>> >>> >>> >> (https://github.com/p2pu/lernanta/blob/master/lernanta/apps/projects/models.py#L270/)
>> >>> >>> >> that in turn gets called in the create view
>> >>> >>> >>
>> >>> >>> >>
>> >>> >>> >>
>> >>> >>> >> (https://github.com/p2pu/lernanta/blob/master/lernanta/apps/projects/views.py#L189)
>> >>> >>> >>
>> >>> >>> >> I'm not sure at what other places SendNotification tasks are
>> >>> >>> >> created,
>> >>> >>> >> but you can grep to code base to find out. grep -r
>> >>> >>> >> SendNotification *
>> >>> >>> >>
>> >>> >>> >> The fact that the places where SendNotification tasks are
>> >>> >>> >> created
>> >>> >>> >> are
>> >>> >>> >> spread all over isn't a problem in itself, but at the moment
>> >>> >>> >> checks
>> >>> >>> >> for notification preference also seem spread out a bit. For
>> >>> >>> >> example
>> >>> >>> >> task updates
>> >>> >>> >>
>> >>> >>> >>
>> >>> >>> >> (https://github.com/p2pu/lernanta/blob/master/lernanta/apps/content/models.py#L234).
>> >>> >>> >> It also makes it difficult to implement batching of
>> >>> >>> >> notifications.
>> >>> >>> >>
>> >>> >>> >> Than being said, I think that notification may be one of the
>> >>> >>> >> first
>> >>> >>> >> parts of Lernanta that we can put into a separate module
>> >>> >>> >> without
>> >>> >>> >> too
>> >>> >>> >> much surgery on the whole code base.
>> >>> >>> >>
>> >>> >>> >> Cheers
>> >>> >>> >> d
>> >>> >>> >>
>> >>> >>> >> On Fri, Jun 1, 2012 at 10:28 AM, Jos Flores
>> >>> >>> >> <josmasflores at gmail.com>
>> >>> >>> >> wrote:
>> >>> >>> >>> Hey Dirk,
>> >>> >>> >>>
>> >>> >>> >>> I know nothing about the topic but curious about where the
>> >>> >>> >>> notifications code is and how it works... could you point me
>> >>> >>> >>> towards
>> >>> >>> >>> it?
>> >>> >>> >>>
>> >>> >>> >>> cheers,
>> >>> >>> >>> José
>> >>> >>> >>>
>> >>> >>> >>>
>> >>> >>> >>> On 1 June 2012 09:01, Dirk Uys <dirk at p2pu.org> wrote:
>> >>> >>> >>>> Hi
>> >>> >>> >>>>
>> >>> >>> >>>> I've been thinking a little bit about two way interaction for
>> >>> >>> >>>> messaging and notifications. On the software side of things I
>> >>> >>> >>>> don't
>> >>> >>> >>>> foresee too much trouble, but on the config side of things
>> >>> >>> >>>> I'm a
>> >>> >>> >>>> little uncertain.
>> >>> >>> >>>>
>> >>> >>> >>>> Our primary MX record points to google, then @lists.p2pu.org
>> >>> >>> >>>> points
>> >>> >>> >>>> to
>> >>> >>> >>>> the service we use for the mailing list. I guess we could use
>> >>> >>> >>>> something like @site.p2pu.org? I see that github uses
>> >>> >>> >>>> @reply.github.com with something that looks like a one time
>> >>> >>> >>>> token.
>> >>> >>> >>>>
>> >>> >>> >>>> Is there any specific mail server that anyone would suggest?
>> >>> >>> >>>> I
>> >>> >>> >>>> have
>> >>> >>> >>>> always skipped the talk about mail servers when reading linux
>> >>> >>> >>>> sysadmin
>> >>> >>> >>>> docs :)
>> >>> >>> >>>>
>> >>> >>> >>>> Cheers
>> >>> >>> >>>> d
>> >>> >>> >>>>
>> >>> >>> >>>> ps. I apologize for the thinking as I type email
>> >>> >>> >>>> _______________________________________________
>> >>> >>> >>>> p2pu-dev mailing list
>> >>> >>> >>>> p2pu-dev at lists.p2pu.org
>> >>> >>> >>>> http://lists.p2pu.org/mailman/listinfo/p2pu-dev
>> >>> >>> >>> _______________________________________________
>> >>> >>> >>> p2pu-dev mailing list
>> >>> >>> >>> p2pu-dev at lists.p2pu.org
>> >>> >>> >>> http://lists.p2pu.org/mailman/listinfo/p2pu-dev
>> >>> >>> >> _______________________________________________
>> >>> >>> >> p2pu-dev mailing list
>> >>> >>> >> p2pu-dev at lists.p2pu.org
>> >>> >>> >> http://lists.p2pu.org/mailman/listinfo/p2pu-dev
>> >>> >>> > _______________________________________________
>> >>> >>> > p2pu-dev mailing list
>> >>> >>> > p2pu-dev at lists.p2pu.org
>> >>> >>> > http://lists.p2pu.org/mailman/listinfo/p2pu-dev
>> >>> >>> _______________________________________________
>> >>> >>> p2pu-dev mailing list
>> >>> >>> p2pu-dev at lists.p2pu.org
>> >>> >>> http://lists.p2pu.org/mailman/listinfo/p2pu-dev
>> >>> >>
>> >>> >>
>> >>> >>
>> >>> >>
>> >>> >> --
>> >>> >> Jessy
>> >>> >> http://jessykate.com
>> >>> >>
>> >>> >>
>> >>> >> _______________________________________________
>> >>> >> p2pu-dev mailing list
>> >>> >> p2pu-dev at lists.p2pu.org
>> >>> >> http://lists.p2pu.org/mailman/listinfo/p2pu-dev
>> >>> >>
>> >>> >
>> >>> >
>> >>> >
>> >>> > --
>> >>> > http://reganmian.net/blog -- Random Stuff that Matters
>> >>> >
>> >>> >
>> >>> > _______________________________________________
>> >>> > p2pu-dev mailing list
>> >>> > p2pu-dev at lists.p2pu.org
>> >>> > http://lists.p2pu.org/mailman/listinfo/p2pu-dev
>> >>> >
>> >>> _______________________________________________
>> >>> p2pu-dev mailing list
>> >>> p2pu-dev at lists.p2pu.org
>> >>> http://lists.p2pu.org/mailman/listinfo/p2pu-dev
>> >>
>> >>
>> >>
>> >>
>> >> --
>> >> http://reganmian.net/blog -- Random Stuff that Matters
>> >>
>> >>
>> >> _______________________________________________
>> >> p2pu-dev mailing list
>> >> p2pu-dev at lists.p2pu.org
>> >> http://lists.p2pu.org/mailman/listinfo/p2pu-dev
>> >>
>> >
>> >
>> >
>> > --
>> > Jessy
>> > http://jessykate.com
>> >
>> >
>> > _______________________________________________
>> > p2pu-dev mailing list
>> > p2pu-dev at lists.p2pu.org
>> > http://lists.p2pu.org/mailman/listinfo/p2pu-dev
>> >
>> _______________________________________________
>> p2pu-dev mailing list
>> p2pu-dev at lists.p2pu.org
>> http://lists.p2pu.org/mailman/listinfo/p2pu-dev
>
>
>
>
> --
> http://reganmian.net/blog -- Random Stuff that Matters
>
>
> _______________________________________________
> p2pu-dev mailing list
> p2pu-dev at lists.p2pu.org
> http://lists.p2pu.org/mailman/listinfo/p2pu-dev
>

More information about the p2pu-dev mailing list