Name   : MSC - a simple message submission client
Purpose: Submitting e-mail messages to an SMTP server (MTA or MSA)
Version: 0.42  (16-04-2023)
Status : Freeware
Author : Frank de Bruijn
E-mail : msc@aconet.org
Website: https://aconet.org/tools/#msc


Intro
=====
You can use MSC to send your e-mail to your (provider's) SMTP server. It knows about most authentication protocols and can be used with servers that support secure connections.
It can also use the CrypStor Query protocol to get login data from CrypStor, so you don't have to store usernames and passwords in its settings file. See https://aconet.org/crypstor/ for more information.


Secure connections
==================
MSC needs either R-Comp's old SecureSockets module or ROOL's new AcornSSL module to make secure connections. SecureSockets is outdated and supports only SSL and TLS 1.0, which are no longer considered secure. AcornSSL, which does handle the more recent TLS versions, is rather difficult to get hold of (which is why there's a recent copy in the package...).

SecureSockets cannot be distributed with MSC or downloaded anywhere, but if you use some of R-Comp's applications (like Grapevine, Hermes, Messenger Pro and others), you probably have it already. Put a copy in System:Modules or in MSC's application directory, with the filename Secure so MSC can find it.

The most up to date AcornSSL is currently only available from the Nightly Beta HardDisc4 zip at https://riscosopen.org/content/downloads/common . Inside the zip, look for HardDisc4.!Boot.Resources.!System.350.Modules.Network.URL.AcornSSL and put it in the corresponding directory on your machine. Don't use anything older than May 2020 [1].


Installation
============
Installing the program is a matter of putting it on you harddrive somewhere. It needs SysLog, so if this isn't installed on your machine, get it from https://compton.nu/software/riscos/syslog


Upgrading
=========
Either drop the new distribution over the one existing on your hard drive or put it somewhere else.
If you install it somewhere else and the Settings file was inside the old version, you'll have to copy or move that to the new one.

MSC uses a very simple mechanism to check for updates: it uses the small BASIC program VerCheck to look up the IP address of a certain non existent subdomain on one of my sites. VerCheck decodes this into a version number and compares the result with the current version. If there's a newer version available, it will report this. It will also create an empty file called NV= in the Choices directory and use that as a flag to check whether this particular new version has already been reported.


Configuration
=============
There's no configuration window. It's all done with a good, old fashioned text file. By default this should be a file called <Choices$Write>.MSC.Settings (or <MSC$Dir>.Settings if your machine doesn't have a Choices structure). You can however specify a different file by adding the full name of that file to the !RunImage line in the !Run file or by changing the line that sets the contents of MSC$Settings.
The settings file is made up of entries - just one for most people - that may look somewhat like this:

  Account     Account name                     SHOULD BE UNIQUE!
  Server      smtp.host.topleveldomain
  Port        25
  Recipients  100
  Client      clienthost.topleveldomain
  Outbound    <Mail$Dir>.Outbound
  Envelopes   <Mail$Dir>.Work
  Username    mailsender
  Password    secret
  Sender      e-mailer@host.topleveldomain     DO NOT USE IN DEFAULT ACCOUNT!
  SSL/TLS
  StartTLS
  StuffDots
  Quit
  AcornSSL
  CertHost    host.topleveldomain              FOR ACORNSSL
  ScrapLocks

No matter how many entries you put in the settings file, the first one will always be the default.

The file Example in the application directory contains a slightly more useful example. This file will be copied to be used as the Settings file if you run the program without supplying one of your own first.

The SSL/TLS, StartTLS, StuffDots, Quit, AcornSSL and ScrapLocks keywords don't take arguments. The Server, Client, Outbound and Envelopes keywords are 'sticky', i.e. you don't have to repeat them in any following entries.
Quit, AcornSSL and ScrapLocks are 'system settings'. You can put them anywhere in the settings file, even before the first Account keyword.

If the settings file cannot be opened or read properly, MSC will default to the name of the server in <Inet$SMTPServer>, the client domainname msc.home and port 25. The maximum number of recipients will be 100 (the SMTP default) and the outbound and envelopes directories specified by <Mail$Dir>.Outbound and <Mail$Dir>.Work (i.e. assuming POP format - see below).

Account
-------
You start an entry with the Account keyword, but other than that the order of the keywords doesn't matter.

Make sure the account name is unique! The program will accept more than one account with the same name, but behaviour may become unpredictable.

Server
------
The Server keyword should be followed by the hostname of the SMTP server you want the program to connect to.

Port
----
If the number of the port to use is 25, you may omit this keyword. Similarly, if you specify SSL/TLS or StartTLS, port 465 or 587 respectively will be assumed if you omit it.

Recipients
----------
The Recipients keyword tells MSC how many recipients a single message may have. If there are more, the program will send the message in multiple transactions, each with no more recipients than set here.
The default value according to the SMTP specification is 100. If you want to use this value, you may omit this setting. However, be aware that these days many ISPs will not allow more than 25 or 50 recipients per message.

Client
------
Client is optional and specifies the domainname to present to the server when the SMTP session is started. You can set this to the domain part of your e-mail address i.e. the bit after the @. However, you shouldn't do this if the domain does not belong to you, so don't use hotmail.com or gmail.com. It can also be the hostname of your cable modem or ADSL router or your IP address. In practice SMTP servers don't do much with this information, apart from storing it in a trace field (Received: header).
If you omit this keyword, MSC will use the value of the system variable Inet$HostName (set in the 'Host name' field of your computer's Network/Internet configuration) [2]. If this variable isn't set, MSC will signal an error and refuse to send anything.

Outbound / Envelopes
--------------------
The Outbound and Envelopes keywords tell the program where your mail reader/writer puts its message files. See the Mail file formats section.

You should not (need to) specify these directories more than once. If you do, the behaviour of the program changes. It will no longer try to find the right server/username/password for a sender and offer everything to the currently selected server. The text in the SendType display icon in the bottom right of the main window will change from 'all' to 'single' in this case. This will also happen if you select any other than the default (topmost) from the Accounts menu (click Menu over the second display icon from the top).

In case you occasionally want to use the default account to send everything without checking the sender, you can click on the SendType display icon and the text will toggle from 'all' to 'single' and back. This only works with the default account and only if multiple use of the Outbound keyword doesn't block the 'all' state.

Username / Password
-------------------
Entering a Username and Password will not have any effect if the server doesn't need authentication. The server should mention whether it does in the initial greeting. If it doesn't, no authentication will be attempted. Complain to your provider if this leads to errors.
If you enter a single asterisk as password, MSC will use the given username as code to query CrypStor for login data.

Sender
------
The optional Sender keyword specifies the sender's e-mail address that the server expects, given the username and password. This enables sending mail using the SMTP server of your Hotmail or GMail account without having to switch manually. BT Internet users may find this useful as well.
You leave off the mailbox name (i.e. use @host.tld) to tell MSC all senders from that domain want to use that server.

The default account SHOULD NOT have a Sender keyword! After all, this account is only used if the others aren't suitable. You should not restrict its use.

SSL/TLS / StartTLS
------------------
SSL/TLS is used if the server requires secure connections immediately (that's usually over port 465). StartTLS is started after an initial plain text connection (either port 25 or 587). Don't use both keywords in one entry. If you do, only SSL/TLS will be acted upon.
These keywords used to be SSL (for SSL/TLS) and TLS (for StartTLS). The current keywords are phrases you may find in other e-mail clients. The olders ones are still accepted.

StuffDots
---------
StuffDots makes MSC double a dot at the start of a line, to prevent them being seen as the end of the message text. This is implied for the POP format (see below).
Marcel format message files already have any necessary stuffing done. If you use this format (i.e. no envelopes directory set - see below) you only need this keyword in case your message files don't come with 'pre-stuffed' dots.

Quit
----
Quit tells MSC to immediately check the Outbound directory for messages waiting to be sent and exit after sending them or if there aren't any. You can also set this option by adding the -quit switch to the !RunImage line in the !Run file or to the !Run line in anything calling that.

AcornSSL
--------
With this, MSC will use the new AcornSSL module for all secure connections.

CertHost
--------
If you use this with AcornSSL, the hostname specified here will be used in the secure connection negotiation. Otherwise the server's hostname will be used. Using CertHost can be useful if you happen to know the certificate the server uses carries a different hostname than the server itself (not as uncommon as you might think...).

ScrapLocks
----------
MSC tries to lock message files that could not be sent for some reason. This won't work on RPCEmu's or VRPC's HostFS. Use ScrapLocks to make MSC use (empty) files in <Wimp$ScrapDir>.MSC.Locks instead. You can use this on real RISC OS hardware as well, by the way, which may be more convenient as removing all the locks in one go is simply a matter of deleting the contents of the Locks directory (or the entire Locks directory, provided MSC isn't running).


Mail file formats
=================
There are two main mail file formats in the Acorn/RISC OS world: the one used by Marcel of the ANT Suite and the POP one, also used by applications like POPstar and Posty (from Voyager). The differences between the two are:
- Marcel format uses one single file per message, which includes the SMTP-
  envelope, i.e. it starts with the lines MAIL FROM:<address>, RCPT TO:<address>
  and DATA. With POP format the envelope data is in a small envelope file with
  the same name in a separate directory.
- Marcel format is 'ready to send', i.e. the file has carriage returns, an
  extra dot at the start of any line that starts with a dot and the five byte
  end of message data sequence (carriage return, line feed, dot, carriage
  return, line feed) is already in place. POP message files don't have any of
  these. The sending client is supposed to add them during sending.

MSC supports both formats. It can handle the differences almost completely transparently, the sole exception being the 'dot stuffing' at the start of a line, which is irrelevant if your messages don't have any lines that start with a dot. Missing carriage returns in a Marcel format message file won't bother it either.
It will expect POP format if the Envelopes keyword points to a work directory where the envelope files are placed by the program you write your messages with. A missing Envelopes keyword will tell MSC the mail files are in Marcel format.

Even if you have set an envelopes directory, you can still use a message file in Marcel format. MSC will notice the envelope file is missing, check the message file itself for a Marcel specific header and use that for the SMTP envelope. There has to be either an envelope file or a Marcel header. MSC will not handle a message which has neither or both.
Occasional Marcel type files in a POP setting will be treated as true Marcel files, i.e. if there's a dot at the start of a line, MSC will assume this has been doubled beforehand. If this isn't so, you need to use the StuffDots keyword or make sure the fifth character of the file is a colon. Because of the way the file is scanned, you can do this without causing any problems, as long as the first four characters of the file are still MAIL (not case-sensitive).

If you use an older version of Pluto and can't get it to select sensible transport settings, use the format and directories set for/by the POPstar transport.


Additional format
=================
In addition to the two 'official' formats, there's a third format that can be used to quickly enter messages into a send queue. This format is a bit of a mixture of the other two. It doesn't use separate envelope files, but the pseudo SMTP-envelope header can be quite compact. In fact, something like this is permissible:
  mail: sender@sender.topleveldomain
  r: recipient@recipient.topleveldomain
  ?
The first part of the MAIL FROM keyword must be given in full (case doesn't matter), but the RCPT TO keyword may be abbreviated to its first character, the e-mail addresses don't have to be enclosed in angled brackets and there may be spaces or tabs between the colons - which are required - and the addresses. The line for the DATA keyword can contain anything, as long as it isn't empty.
The message file doesn't need the carriage returns or the termination sequence of the Marcel format and should not have the 'pre-stuffed' dots. The StuffDots keyword is not required, provided the fifth character in the file is a colon.

Be aware this is only about the ENVELOPE!. You still need to create the actual message with normal headers and body. That is, the shortened MAIL FROM and RCPT TO headers from the envelope will not be used as the message's From: and To: headers.


Running MSC / the main window
=============================
Run the program by double clicking the application directory or the !Run file as usual. MSC will not put an icon on the icon bar, but there is a main window with some (unmarked) display fields and a couple of buttons.

Click Send and if there's any file in the outbound directory, it will be sent as a message. Locked files and directories are ignored.
During sending any Bcc headers that may be present will be removed.
If the sending fails - even for just one recipient - MSC will lock the message file and tell you about the problem so you can fix it, unlock the file and try again (or remove the file or whatever). Check the log or the Report file (more about this later).
Cancelling the session does not lock any files.

The five larger display icons show various bits of information, like overall program state (top icon), end results or errors (bottom icon).

Closing the window doesn't quit the program. Use the Quit button, the menu or the taskmanager's Tasks window. If you closed the window, you can reopen it by running the program again. This will not start a new incarnation - just wake up the previous one.

As mentioned earlier, the bottom right display icon (unmarked, but called 'SendType') indicates whether the program will use all available accounts to send the messages or just the single one which is currently selected (visible in the second display icon from the top). MSC will only lookup the senders of messages in the list of senders supplied through the Sender keywords if SendType shows 'all'.


Drag and drop
=============
If you drop a text file on MSC's main window, the program will try to interpret it as a message to send. If the internal SMTP-envelope of the Marcel or additional format is present, this will be used. If it isn't, the program will try to extract the necessary data from the message headers. It will create an envelope file in the envelopes directory or - if this isn't set - its Scrap directory.


Report
======
MSC creates a line in a file called <SysLog$LogDir>.MSC-Report at the start and end of the session and for every transaction. A line with a message filename preceded by a 110 code indicates a message sent without any problems. Any other number indicates some kind of error. There will also be lines with recipient's addresses. These start with either the SMTP code 250 or an error code like 550.
This function was initially used for debugging purposes, but I left it in because of a vague notion of doing something useful with it. It may be removed in a later version of the program, unless the notion becomes less vague...

For now, you can switch it off if you don't want it. Either add a NoReport line to the Settings file or a -noreport switch to the command line.


Disclaimer
==========
This software is provided free of charge and 'as is'. It comes with no guarantees of any kind. I think it does what it's supposed to do, but I accept no responsibility whatsoever for any loss or damage caused by use of or inability to use this software.

That said, if the program fails somehow, do get in touch with me and I'll see if I can fix it.


Error messages
==============
Some possible error messages, both from the main window and the log file.

- DataLoad error - <text>
- No sender
- No recipients
  You've dropped a file on MSC's main window but the program can't read it or
  hasn't been able to extract sender or recipients.

- Settings error - line <number>
- Settings error - <some text>
  An error in the settings file. Check the syntax.
  In case <some text) is 'Missing server', you have not specified any smtp
  server. For 'No client and Inet$Hostname not set' look at your computer's
  Internet settings. The system variable Inet$Hostname really should be set.

- CheckOutbound error - <text>
- No sender
  There are message files in the outbound directory that cannot be read or for
  which the sender cannot be determined.

- CrypStor error - <text>
- CrypStor not running
  Errors to do with CrypStor. The textual message should tell you how to handle
  them.
  As sending e-mails usually is an interactive process, MSC currently does not
  attempt to start CrypStor if it isn't running.

- StartTLS required but not supported
  You've specified StartTLS in the settings, but the server doesn't seem to do
  secure transfers.

- AUTH required but no type supported
  MSC can handle the commonly used authentication types LOGIN, PLAIN and
  CRAM-MD5. Apparently the server is only able to use some other type which the
  program knows nothing about.

- Envelope <filename> not found, not a file, locked or empty
- Message file <filename> has external envelope or cannot be read
  Missing or less than useful envelope file, an envelope file when it isn't
  wanted or file read problems.

- Error sending MAIL FROM - <text>
  The error text 'message has no sender' or 'message is too large' should be
  self explanatory. I/O errors are also possible.

- Error sending RCPT TO - <text>
  Most likely 'message has no recipients'.

- Message has no contents
  This doesn't mean you've tried to send an empty message (which is fine) but
  there's actually nothing there at all, i.e. no headers either (which isn't).

- No data send acknowledgement
- Error sending message text - <text>
- Unexpected response from server
  Unexpected response codes from the server or other errors while sending the
  actual message.

- Secure socket write error
- Not all command data sent
- Connection lost
- Operation timed out
- Error &<number> - <text>
- Handshake error (state <number>)
  Transfer errors. Bad connection? Server problems? Bad certificate?

- Error &818047 - Bad Resolver request
  Resolver acting up. It should really report 'Host not found' or something
  like that. In a TaskWindow, gethost nonexistinghost.com produces:
    Looking up hostname 'nonexistinghost.com'
    gethost: gethostbyname('nonexistinghost.com') failed.
    Report: Handle is either illegal or has been closed
  This happens on RPCEmu, so that could be part of the problem (although RPCEmu
  does not use the host's internet stack as VRPC does).

- Buffer overflow in AUTH routines at #<number>
- Allocation failed
- Template too large
- Not indirected
  Program errors that really shouldn't occur...

With these descriptions it should be clear enough how to handle most of these errors. In case it isn't, get in touch with me, quoting the error that occurred and as much extra information as possible.


--------
[1] ROOL is using a downright useless version numbering system for the AcornSSL
    module. There are currently about half a dozen of version 1.05 and sixteen
    of 1.06, but they're all different and at least one of the 1.05s is badly
    broken. You need to look at the 'mbedTLS' bit in the help string to tell
    them apart (and even that doesn't always work...).
    Also, NEVER use the 1.05 version that has an MD5 checksum of
    138542e96aeb171ef81dbac5033622e0.

[2] Strictly speaking using the hostname isn't correct as this will never
    resolve to the right IP address (which it should, according to RFC5321).
    However, POPstar has always done it as well and the alternative would be to
    add code to determine the actual IP address of your internet connection and
    I'm not prepared to do that. :-)
