Skip to content
A scene from Ireland

2004

  • in Linux
  • 1 min read

Event monitoring

Events happen in raid arrays, such as device failure and resync. It would be nice to know when they happen. polling /proc/mdstat is one option, but it isn't perfect.

  • in Opinion
  • 2 min read

Address rewriting

We want to re-write addresses and recognise rewritten addresses.

There are two situations where we want to re-write addresses.

BULLET - If we trust the source of a return address, we want to re-write it when sending the mail to be signed and to look like it comes from us. If it really comes from a local domain that we aren't the only MX for, then rewriting isn't appropriate. I probably need to refine 'local domain' into those that we control completely and those that we spool and forward mail for. BULLET - When we are forwarding mail through the virus checker, we want to re-write the sender and recipient addresses. This rewriting should be different from the other rewriting and should be transparent to the rest of the mail system.

Recognising re-written address needs to happen for all incoming mail - the to address can always be encapsulated.

Also, we want to recognise special recipient address that are coming from forwarders. e.g. if the localpart contains an equals sign, then we check if the domain which follows that equals is a reliable source of the mail.

The rewriting of return-path should keep the original domain for local domains, and use cse.unsw.edu.au for all others.

In the first case we can use SES= to flag the address.

For others, SRS= seems better.

For virus re-writing, VRF and VRT for the virus from and to addresses.

If we get mail from a VRF and to a VRT, we accept it. If it is to a VRF and from <>, then it has to go to postmaster. Otherwise we reject it.

  • in Opinion
  • 1 min read

SPF-Received tagging

smtp-recv should tag all arriving email with SPF-Received.

If the mail is to an address (and only one address) which contains an equals sign followed by atleast one period, we use that domain for SPF checking.

Otherwise if the mail is from <>, then we assume FAIL if the to address wasn't encapsulated, and PASS if it was.

Otherwise we perform an SPF check on the MAIL FROM address.

  • in Opinion
  • 1 min read

Authenticate incoming mail

All incoming mail that claims to be from a local address must, eventually, be authenticated. I.e we must have some trace of who sent it.

There are three ways to be authentic:

  • Come from a local trusted machine that supports identd
  • Come over SSL with a username and password
  • Come over SSL with a certificate that we have signed

In general, any authenticated user may use any address. We need to be sure that more widely accessable users such as w3serv which runs our webmail service cannot be abused.

The first two options are already available. We need a system for handing out signed SSL certificates. The common name should be something like mailfrom:neilb@cse.unsw.edu.au. This still allows mail from any local address, but it identifies the source.

Once this is inplace and there are some FAQs that describe it, we need to start identifying people who aren't being authenticated and need to be. Some simple logging in the smtp receiver should achieve this easily.

When the log shows that sufficiently few people (preferably none) are not authenticated properly, we disable non-authenticated users for external addresses completely. For internal addresses we need to accept them but with an SPF tag indicating probable junk.

  • in Linux
  • 2 min read

auto-correcting read errors

Write errors must always fail a device as even if there was a cable problem rather than a media problem, the drive will be inconsistant with the array after the write failure and so cannot be trusted.

However read errors do not have to be fatal. If the device can over-write a bad sector, or remap to elsewhere, then it makes sense to regenerate the data from redundant info and re-write.

  • in Linux
  • 3 min read

RAID10

While it is quite possible to provide a RAID10 array by making some RAID1 arrays and combining them with RAID0, this approach has some problems.

  • in Linux
  • 3 min read

Better read-ahead management and other file-open issues

NFS (prior to V4) does not have a concept of opening and closing a file. The Linux VFS does include that concept. So the NFS server has to open the file before a READ or WRITE operation, and close it afterwards.

Some funcitonality in filesystems and in the VFS assumes that the open/close requests that are passed down correspond to opens and closes by the application, and that a particular "file" structure belongs to a particular application (possibly a group of processes). Thus he current open/IO/close approach does not give good information to filesystems