[NCSG-EC] Mailing list issues

[Tapani Tarvainen]

Dear all,

As you may recall we talked briefly about NCSG's mailing list problems
in Prague. There are two distinct issues we should address:

First, NCSG-DISCUSS messages tend to be eaten by spam filters,
especially by Google.

Second, having NCSG's mailing lists hosted on a volunteer basis
is not ideal in the long term, as there's no guarantee of
continuity.

The first is more urgent. The problem there is that Google
(and some others) have tightened their spam filters in a way
that's somewhat difficult for mailing lists to handle.

For the technically inclined, Google wants incoming mail
to have either SPF or DKIM set up. For some reason, Syracuse
university (where ncsg-discuss is hosted) has not set up
DKIM at all and their SPF setting seems to be broken.

I have not contacted Syracuse support yet. I am a bit wary
of doing so given that we have contract or indeed any relation
with them, so they might just tell us to move our list away.
At least I'd like to have a backup plan ready if that happens.

Indeed given what I've seen at some other universities I fear
Syracuse may run the service down at some point. So a backup
plan would be good in any case.

Anyway, I can see three options here:

(1) Wait and hope Syracuse will at least fix their SPF setup.
    This is reasonably likely to happen sooner or later, but
    it's not guaranteed and may take some time. Also Google &co
    may start insisting on both SPF and DKIM some day.

(2) Contact Syracuse support. That might get them to fix their
    SPF sooner or maybe even set up DKIM. Or it might not.

(3) Move ncsg-discuss somewhere else. The obvious choice would be
    lists.ncsg.is, which is hosted by Electronic Frontier Finland
    (Effi) and where all other ncsg lists are. In the long term,
    however, it would be better to have ncsg lists somewhere under
    ncsg's own control.

If we go for (3), we could either first move ncsg-discuss to Effi and
later all ncsg lists when we find a good long-term solution, or leave
it where it is until we have the long-term solution decided.

On the face of it the latter would seem to be less work, but the
difference may not be all that big: moving all lists from one place to
another will be just about as hard whether there's one list more or
less, the extra work caused by different software at Syracuse will be
similar in either case. And the spam filter problems at Syracuse will
probably persist while we're looking for the long-term solution, and
that may take surprisingly long time.

One point to consider: moving ncsg-discuss away from Syracuse will
change the list's address and our members will have to learn to use
the new one. It should be possible, however, to set up temporary
forwarder from the old list to mitigate this. Moving the lists
away from Effi would not have this issue because (presumably)
we'd move the address lists.ncsg.is as well.

Some considerations when looking for new mailing list hosting:

* We need to keep old archives.

* We would like to have a reasonably easy way to remove messages
  or parts of them (like attachments) from archives if need be.

* We would like to keep our own domain (lists.ncsg.is) in the
  list addresses.

* It would be nice to have a solution that synchronizes
  (semi)automatically with our member database, whatever that
  will be in the future.

* It would be ideal to have a solution for both member database
  and mailing lists as one package. (CiviCRM theoretically could.
  In practice... well.)

* We want to have a solution that is sustainable and doesn't
  depend too much on volunteers and avoids key person dependency
  problems.

* We want a solution that generally works well. That is, spam
  protection and the like are (and are kept) up to data, good list
  management interface &c.

* We want something reasonably cheap. Like, something we can get
  ICANN to pay for.

I've probably missed something but I trust you all feel free
to add things that come to your mind.

Finding a good solution may not be easy and implementation may also
take a fairly long time, especially if we try to move the member
database at the same time.

So, question: Do we want to fix the spam filter issue ASAP or leave it
until we have a comprehensive long-term solution?

-- 
Tapani Tarvainen