[p2pu-dev] Two way communication for notifications and messaging

Stian Håklev shaklev at gmail.com
Mon Jun 4 20:17:17 UTC 2012


Well, I guess every single message is sent out with a unique token, like

reply-34093049304 at p2pu.org

the combination of this token, and the sender e-mail address should be
enough authentication?

Or do you mean how do we know that the HTTP posts from cloudmailin actually
come from cloudmailin?

Apart from the possible vicissitudes of handling email ourselves (which I
have no experience with, but it always seems more complex than what you'd
think - although perhaps that's more for sending email than for receiving),
it really doesn't seem all that difficult to me.

For each notification e-mail, generate a unique ID, and store that ID and
the e-mail addresses that received the notification in a database. When you
receive a reply, look up the ID, check that e-mail match, and look up where
to append the reply.

Stian

On Mon, Jun 4, 2012 at 4:13 PM, Jos Flores <josmasflores at gmail.com> wrote:

> How does this handle authorisation though? I see they do have docs but
> I'm not following... Would it be related to the email account the user
> signed up on the site with?
> http://docs.cloudmailin.com/features/callbacks/#authorization-callbacks
>
> They even have a Django app all ready:
> https://github.com/CloudMailin/django-cloudmailin      ...tempting
> indeed! :)
>
> cheers,
> José
>
>
> On 4 June 2012 18:51, Jessy Kate Schingler <jessy at jessykate.com> wrote:
> > that's cool dirk i've not heard of cloudmailin!
> >
> > looking at their site it seems like a pretty neat service in general, but
> > i;m not sure how much it would buy us in terms of reducing the work we
> have
> > to do on the server side. doing it ourselves i guess we would write a
> script
> > that parses incoming mails in the postfix maildir or exim equivalent
> (which
> > cloudmailin would take care of). but we still have to disambiguate
> responses
> > into specific accounts/discussion threads/models.
> >
> > anyway, my sense is that it wouldn't save us a ton of effort, and would
> > increase dependencies on external services, but curious what others
> think.
> >
> > other thoughts?
> >
> >
> > On Mon, Jun 4, 2012 at 8:53 AM, Stian Håklev <shaklev at gmail.com> wrote:
> >>
> >> To get a sense of the traffic levels, how many messages are posted every
> >> month on P2PU right now? (Of course, we'd then have to estimate roughly
> what
> >> percentage of those messages are likely to be posted through email
> rather
> >> than online).
> >>
> >> Stian
> >>
> >>
> >> On Mon, Jun 4, 2012 at 11:22 AM, Dirk Uys <dirk at p2pu.org> wrote:
> >>>
> >>> Googling around a bit I found http://www.cloudmailin.com/
> >>>
> >>> They turn your email into an http post to your website. I like the
> >>> idea, it keeps the email stuff separate from the messaging logic.
> >>>
> >>> They also have special pricing options for non-profits.
> >>>
> >>> The alternative would be to setup our own mail server and use some
> >>> python bindings for mail and a celery task. There's poplib and imaplib
> >>> for python.
> >>>
> >>> Any thoughts
> >>>
> >>> Cheers
> >>> d
> >>>
> >>> On Sat, Jun 2, 2012 at 1:02 AM, Stian Håklev <shaklev at gmail.com>
> wrote:
> >>> > A lot of emails disregard anything after a + in an email user name,
> >>> > like
> >>> > GMail which will send all mail to shaklev+test at gmail.com to
> >>> > shaklev at gmail.com. So if the server receives all mail to
> reply+dfsfd to
> >>> > the
> >>> > reply address, and the reply address is able to strip this out of the
> >>> > header, and then know where to put it, based on the unique ID, we
> >>> > should be
> >>> > good to go.
> >>> >
> >>> > I am sure there are services and APIs out there for doing this as
> well,
> >>> > but
> >>> > they are probably expensive for the kind of volume (although we'd
> have
> >>> > a lot
> >>> > less replies by email than we have email notifications).
> >>> >
> >>> > For such a common problem, you'd think there'd be a Django module or
> at
> >>> > least a Python implementation out there somewhere as well. But I
> don't
> >>> > know
> >>> > how much of the challenge is in the Lernanta codebase, and how much
> is
> >>> > the
> >>> > email server etc.
> >>> >
> >>> > Stian
> >>> >
> >>> >
> >>> > On Fri, Jun 1, 2012 at 1:44 PM, Jessy Kate Schingler
> >>> > <jessy at jessykate.com>
> >>> > wrote:
> >>> >>
> >>> >> really cool dirk.
> >>> >>
> >>> >> i dont have a particular mail server recommendation though have
> >>> >> personally
> >>> >> done more with postfix than exim.
> >>> >>
> >>> >> i just did a quick search for "comments reply by email" and noticed
> >>> >> that
> >>> >> wordpress implemented this last year. obviously they're php but it
> >>> >> might be
> >>> >> interested to take a look at the WP codebase to see how they
> >>> >> implemented it.
> >>> >>
> >>> >> looking at facebook notifications, they use emails structured like
> so:
> >>> >> update+ohlp1evf at facebookmail.com
> >>> >>
> >>> >> so i guess update@ is their updates email address and the hash
> let's
> >>> >> them
> >>> >> disambiguate to a specific conversation thread via some kind of
> script
> >>> >> internally. i think this would be a fun/interesting thing to
> >>> >> contribute to
> >>> >> if it gets going, but know my time will be somewhat limited this
> >>> >> summer with
> >>> >> internship work.
> >>> >>
> >>> >> jessy
> >>> >>
> >>> >>
> >>> >> On Fri, Jun 1, 2012 at 6:26 AM, Dirk Uys <dirk at p2pu.org> wrote:
> >>> >>>
> >>> >>> Thanks, having a look at ack atm!
> >>> >>>
> >>> >>> On Fri, Jun 1, 2012 at 3:07 PM, Jos Flores <josmasflores at gmail.com
> >
> >>> >>> wrote:
> >>> >>> > Dirk, thanks for the walkthrough, it does make sense.
> >>> >>> >
> >>> >>> > I've noticed you recommend grep... have you tried ack? it's
> really
> >>> >>> > cool!
> >>> >>> > I use grep very little since I discovered it :)
> >>> >>> >
> >>> >>> > cheers,
> >>> >>> > José
> >>> >>> >
> >>> >>> > On 1 June 2012 09:57, Dirk Uys <dirk at p2pu.org> wrote:
> >>> >>> >> No problem. At the moment the notification code isn't in one
> place
> >>> >>> >> (something we can work on).
> >>> >>> >>
> >>> >>> >> I'm going to start explaining from the point where the email is
> >>> >>> >> sent
> >>> >>> >> to the user and work my way backwards to where notifications are
> >>> >>> >> generated. Please tell me if this ends up being confusing
> >>> >>> >>
> >>> >>> >> Emails are sent to a user by calling
> >>> >>> >> django.contrib.auth.models.User.email_user
> >>> >>> >>
> >>> >>> >>
> >>> >>> >> (
> https://docs.djangoproject.com/en/dev/topics/auth/#django.contrib.auth.models.User.email_user
> )
> >>> >>> >>
> >>> >>> >> email_user() is called by an async celery task SendNotification
> >>> >>> >>
> >>> >>> >>
> >>> >>> >> (
> https://github.com/p2pu/lernanta/blob/master/lernanta/apps/users/tasks.py#L25
> )
> >>> >>> >>
> >>> >>> >> SendNotification tasks can be created in multiple places:
> >>> >>> >>
> >>> >>> >> There is a signal
> >>> >>> >>
> >>> >>> >>
> >>> >>> >> (
> https://github.com/p2pu/lernanta/blob/master/lernanta/apps/content/models.py#L245
> )
> >>> >>> >> that fires when tasks are created or modified. This signal then
> >>> >>> >> calls
> >>> >>> >> a function called send_email_notification
> >>> >>> >>
> >>> >>> >>
> >>> >>> >> (
> https://github.com/p2pu/lernanta/blob/master/lernanta/apps/content/models.py#L213
> )
> >>> >>> >> that creates SendNotifications tasks.
> >>> >>> >>
> >>> >>> >> Another example is sent_creation_notification
> >>> >>> >>
> >>> >>> >>
> >>> >>> >> (
> https://github.com/p2pu/lernanta/blob/master/lernanta/apps/projects/models.py#L270
> )
> >>> >>> >> thats part of projects.models.Project. This function is called
> >>> >>> >> from
> >>> >>> >> the create function
> >>> >>> >>
> >>> >>> >>
> >>> >>> >> (
> https://github.com/p2pu/lernanta/blob/master/lernanta/apps/projects/models.py#L270/
> )
> >>> >>> >> that in turn gets called in the create view
> >>> >>> >>
> >>> >>> >>
> >>> >>> >> (
> https://github.com/p2pu/lernanta/blob/master/lernanta/apps/projects/views.py#L189
> )
> >>> >>> >>
> >>> >>> >> I'm not sure at what other places SendNotification tasks are
> >>> >>> >> created,
> >>> >>> >> but you can grep to code base to find out. grep -r
> >>> >>> >> SendNotification *
> >>> >>> >>
> >>> >>> >> The fact that the places where SendNotification tasks are
> created
> >>> >>> >> are
> >>> >>> >> spread all over isn't a problem in itself, but at the moment
> >>> >>> >> checks
> >>> >>> >> for notification preference also seem spread out a bit. For
> >>> >>> >> example
> >>> >>> >> task updates
> >>> >>> >>
> >>> >>> >> (
> https://github.com/p2pu/lernanta/blob/master/lernanta/apps/content/models.py#L234
> ).
> >>> >>> >> It also makes it difficult to implement batching of
> notifications.
> >>> >>> >>
> >>> >>> >> Than being said, I think that notification may be one of the
> first
> >>> >>> >> parts of Lernanta that we can put into a separate module without
> >>> >>> >> too
> >>> >>> >> much surgery on the whole code base.
> >>> >>> >>
> >>> >>> >> Cheers
> >>> >>> >> d
> >>> >>> >>
> >>> >>> >> On Fri, Jun 1, 2012 at 10:28 AM, Jos Flores
> >>> >>> >> <josmasflores at gmail.com>
> >>> >>> >> wrote:
> >>> >>> >>> Hey Dirk,
> >>> >>> >>>
> >>> >>> >>> I know nothing about the topic but curious about where the
> >>> >>> >>> notifications code is and how it works... could you point me
> >>> >>> >>> towards
> >>> >>> >>> it?
> >>> >>> >>>
> >>> >>> >>> cheers,
> >>> >>> >>> José
> >>> >>> >>>
> >>> >>> >>>
> >>> >>> >>> On 1 June 2012 09:01, Dirk Uys <dirk at p2pu.org> wrote:
> >>> >>> >>>> Hi
> >>> >>> >>>>
> >>> >>> >>>> I've been thinking a little bit about two way interaction for
> >>> >>> >>>> messaging and notifications. On the software side of things I
> >>> >>> >>>> don't
> >>> >>> >>>> foresee too much trouble, but on the config side of things
> I'm a
> >>> >>> >>>> little uncertain.
> >>> >>> >>>>
> >>> >>> >>>> Our primary MX record points to google, then @lists.p2pu.org
> >>> >>> >>>> points
> >>> >>> >>>> to
> >>> >>> >>>> the service we use for the mailing list. I guess we could use
> >>> >>> >>>> something like @site.p2pu.org? I see that github uses
> >>> >>> >>>> @reply.github.com with something that looks like a one time
> >>> >>> >>>> token.
> >>> >>> >>>>
> >>> >>> >>>> Is there any specific mail server that anyone would suggest? I
> >>> >>> >>>> have
> >>> >>> >>>> always skipped the talk about mail servers when reading linux
> >>> >>> >>>> sysadmin
> >>> >>> >>>> docs :)
> >>> >>> >>>>
> >>> >>> >>>> Cheers
> >>> >>> >>>> d
> >>> >>> >>>>
> >>> >>> >>>> ps. I apologize for the thinking as I type email
> >>> >>> >>>> _______________________________________________
> >>> >>> >>>> p2pu-dev mailing list
> >>> >>> >>>> p2pu-dev at lists.p2pu.org
> >>> >>> >>>> http://lists.p2pu.org/mailman/listinfo/p2pu-dev
> >>> >>> >>> _______________________________________________
> >>> >>> >>> p2pu-dev mailing list
> >>> >>> >>> p2pu-dev at lists.p2pu.org
> >>> >>> >>> http://lists.p2pu.org/mailman/listinfo/p2pu-dev
> >>> >>> >> _______________________________________________
> >>> >>> >> p2pu-dev mailing list
> >>> >>> >> p2pu-dev at lists.p2pu.org
> >>> >>> >> http://lists.p2pu.org/mailman/listinfo/p2pu-dev
> >>> >>> > _______________________________________________
> >>> >>> > p2pu-dev mailing list
> >>> >>> > p2pu-dev at lists.p2pu.org
> >>> >>> > http://lists.p2pu.org/mailman/listinfo/p2pu-dev
> >>> >>> _______________________________________________
> >>> >>> p2pu-dev mailing list
> >>> >>> p2pu-dev at lists.p2pu.org
> >>> >>> http://lists.p2pu.org/mailman/listinfo/p2pu-dev
> >>> >>
> >>> >>
> >>> >>
> >>> >>
> >>> >> --
> >>> >> Jessy
> >>> >> http://jessykate.com
> >>> >>
> >>> >>
> >>> >> _______________________________________________
> >>> >> p2pu-dev mailing list
> >>> >> p2pu-dev at lists.p2pu.org
> >>> >> http://lists.p2pu.org/mailman/listinfo/p2pu-dev
> >>> >>
> >>> >
> >>> >
> >>> >
> >>> > --
> >>> > http://reganmian.net/blog -- Random Stuff that Matters
> >>> >
> >>> >
> >>> > _______________________________________________
> >>> > p2pu-dev mailing list
> >>> > p2pu-dev at lists.p2pu.org
> >>> > http://lists.p2pu.org/mailman/listinfo/p2pu-dev
> >>> >
> >>> _______________________________________________
> >>> p2pu-dev mailing list
> >>> p2pu-dev at lists.p2pu.org
> >>> http://lists.p2pu.org/mailman/listinfo/p2pu-dev
> >>
> >>
> >>
> >>
> >> --
> >> http://reganmian.net/blog -- Random Stuff that Matters
> >>
> >>
> >> _______________________________________________
> >> p2pu-dev mailing list
> >> p2pu-dev at lists.p2pu.org
> >> http://lists.p2pu.org/mailman/listinfo/p2pu-dev
> >>
> >
> >
> >
> > --
> > Jessy
> > http://jessykate.com
> >
> >
> > _______________________________________________
> > p2pu-dev mailing list
> > p2pu-dev at lists.p2pu.org
> > http://lists.p2pu.org/mailman/listinfo/p2pu-dev
> >
> _______________________________________________
> p2pu-dev mailing list
> p2pu-dev at lists.p2pu.org
> http://lists.p2pu.org/mailman/listinfo/p2pu-dev
>



-- 
http://reganmian.net/blog -- Random Stuff that Matters
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.p2pu.org/pipermail/p2pu-dev/attachments/20120604/bf07c983/attachment-0001.html>


More information about the p2pu-dev mailing list