Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Zen Section
id739853429

The Email importing function accessible at System>Batch process menu>Data management> Data import/export>Email importing provides the ability to create new questions from emails arriving at specified email addresses, and create updates to existing questions from emails arriving at the RefTracker Service email address. In other words this function means RefTracker can assimilate new questions and question updates arriving by email at the email addresses specified here. When Email importing is set up in System>Batch process menu>Background processing, Email importing from the accounts that you set up here (in System>Batch process menu>Data Management>Data import/export>Email importing) will happen on a regular and automatic basis using the parameters (such as frequency of pickup) defined there. Your System Administrator (or any other email address/es defined in Parameter 1.3) is automatically advised if any email cannot be automatically processed so that they can manage these exceptions manually.

Despite being able to use this function to automatically accept questions sent by email, it is to your advantage to have clients use RefTracker forms to submit questions wherever possible. The reference interview provided by a RefTracker form ensures that you receive the information you need so that follow up interviews are minimised. When a client sends you an email they only send the information that they think is important resulting in a high percentage of follow up interviews being required. Further, when you receive an email, only minimal information can be collected for statistical purposes. For both of these reasons, when implementing RefTracker, you should be replacing all places that advertise your organisation's email address, with links to RefTracker forms where possible, in order to minimise the number of questions arriving by email.

Nevertheless, we recognise that many organisations, particularly high intensity ones like legal libraries, need to operate by picking up new requests from emails, and this email importing functionality provides a comprehensive solution for organisations that operate in that way.

Most RefTracker systems will utilise this email importing functionality to some extent, even if they only use the ability to automatically process update emails arriving back at the RefTracker service email address!

A common use of the new request import functionality is to accept new requests that come in as emails from other applications, like online databases.

Service email account and New request email accounts

RefTracker email importing handles two sorts of importing:

- "Service account" where the emails coming into this account will be replies and notifications in relation to the emails being sent out by RefTracker (it's the email address specified in your RefTracker parameter 1.2). Where an email comes into this account with a valid RefTracker question number in the subject line, a note containing the name, email address, subject line and body of that email,plus any attachments, will be added to the History of that question number, and the staff member currently allocated that question, if any, will be sent an email containing the text of the email and a link to the question in RefTracker so they can work on it. Where the email does not have a valid question number in the subject line, your System Administrator (or any other email address/es defined in Parameter 1.3) is automatically advised by email so that they can manage these exceptions manually (a one click option allowing any new requests arriving at this address to be resubmitted as a new request is included in that email) .

- "New Request" where the emails coming into this account will all be used to create new questions. For each email received into this account a new RefTracker question will be created that contains the subject line followed by the body of that email as the Question text, the sender's email address in the Client email field, their email name into the Client name field (if a name has been supplied), and any cc: addresses are saved into the RefTracker cc: field. Any attachments are added as question attachments.  The new question is created using the Request form that you define in the set up, so any other data specified in that Request form such as Request type or Location will also be included in the question, and the processing specified by the form, such as sending a confirmation email, will be automatically undertaken. Your System Administrator (or any other email address/es defined in Parameter 1.3) is automatically advised if any email cannot be imported so that they can manage these exceptions manually (if the message requires action).

Sometimes emails that you want to become new questions in RefTracker, arrive into email accounts that are not being monitored by RefTracker such as an account being triaged by staff or even emails arriving into Staff personal email accounts.  These emails can be forwarded to RefTracker as an attachment/s to an email to an email account that is being monitored by RefTracker.  Click here and go to the section about "Forward as an attachment" for details of how to do this.

Setting up email importing for use

To set up the email accounts that can be monitored by RefTracker, go to System>Batch processes menu>Data import/Export>Email importing. 



Email is retrieved from the email addresses that you define here using the POP3 or IMAP (including IMAP subfolders) or SMTP drop folder protocols.
As emails are picked up by RefTracker from these email accounts they are automatically deleted from that email account and copied into RefTracker. For this reason any email address used for this purpose must be an email address that is dedicated to this purpose only.

To use this function you will have to arrange with your email system manager to have POP3 or IMAP set up on the email address/es that you choose to use, or to have them set up as SMTP drop folders. If you have a hosted system you will need to use an email account that is valid for the hosted system. Altarama owns the reference-service.info, and libraryresearch.info domains so we can provide generic addresses for this purpose. If you have as secure hosted system, Altarama will set up the email importing accounts that use the domain used by your secure service.

Why use email importing

Unless your system is used for DeskStats only, you will want to use this Email importing function to monitor your RefTracker service email address (as defined by parameter 1.2) so that replies and non-delivery notices and other emails arriving at that address can be automatically added as notes to the question to which they belong. You will need to setup an entry here for your Service email address (and ensure that the Import email background task is set up for this account when you want it to start running automatically). Be sure that your Service email address is not being used for any other purpose than RefTracker outgoing emails.
If you have new questions arriving into your organisation by email, and the senders of those emails cannot be persuaded to use a RefTracker form instead, you will need to set up an entry in this page for each email address where email will arrive, and have POP3, IMAP or SMTP drop folders set up for each of those email addresses (for hosted customers this can be one of your internal email addresses that is aliases to an Altarama provided account). If your IT department does not allow one of these protocols, it can be one of your email accounts that is associated with a POP3 or IMAP account provided by Altarama. RefTracker picks up all of the questions that arrive into these accounts and turns them into new RefTracker questions, so these email addresses need to be dedicated to new questions for RefTracker. You can do that by publishing the specific RefTracker monitored email addresses.

Note that if you "forward" an email, the person who forwarded the email will become the client, instead of the actual client – so don't forward emails directly to these accounts unless you are prepared to manually change the client contact details.  Instead use the RefTracker Import email by forwarded attachment process. See the section called "Emails forwarded as attachments to RefTracker email importing" in the How questions for follow up arrive in your system help page, for details of how this can be done manually or with just a few clicks using a Email client workflow function such as Outlook's Quick Step.

Safety of using email importing

RefTracker will import all of the email arriving at the email accounts that you designate to be monitored by RefTracker.

This means that spam arriving at those email account/s WILL be imported into RefTracker – the level of spam arriving will be determined by the effectiveness of the email spam filter in use at your organisation (or if your email address is provided by Altarama, by our up to date Barracuda spam filter with the latest filtering patches always applied). Not every spam email can be captured by spam email filters so you will definitely have some spam questions being created in RefTracker if you use email importing.

RefTracker can apply content filtering (parameter 0.20) that will result in questions created from spam emails being automatically marked as spam in RefTracker, but we have found that it will also identify real questions as spam. Our experience is that a human eye can easily identify spam questions displayed in the Open questions screen, and so the most efficient way to deal with any spam that gets through is to quickly delete the RefTracker questions created by spam emails.

Further it is possible for spam emails to get through your spam filter, yet still include code that might cause a security issue with applications like RefTracker.  RefTracker addresses this risk through the following:

  • For the safety of all our customer's systems RefTracker uses a content filter to ensure that any control characters in an email that might indicate inclusion of program code, are converted to their character code equivalents before being stored in RefTracker. This ensures that any malicious code imported from an email is not stored in a way that would allow it to be used as code.
  • There is also a parameter controlled function that sends an email to the parameter 1.3 email address instead of importing an email, when the imported email has more that the Parameter 0.12 number of email addresses in any one of the To:, cc: or bcc: fields. Experience tells us that any more than 20 (default) such email addresses, means that the email is most likely spam.  Note that if the email was sent to the RefTracker email importing address as a member of a long cc: list, it will arrive with your address in the to: field and with nothing in the cc: field.
  • Further we recommend that you set the Request form controlling your email importing to NOT send confirmation emails.  This will ensure that when spam emails do enter your RefTracker system, you are not encouraging more such spam, by the confirmation email confirming they have reached a valid email address.  Senders of emails already have a copy of the email that they sent as their evidence of when the request was sent, so a confirmation email is not really necessary for imported emails.

Handling HTML formatted emails

RefTracker can import all types of email - including text and HTML formatted emails. HTML formatted emails will be "simplified" before being imported into the RefTracker question field, to prevent any malicious code being picked up from HTML formatted emails. HTML formatted emails can contain formatting that is important to the understanding of the email, that could be removed by the simplification process, so we make the original source email available to be viewed from an [original email] at the end of the question summary.

Images embedded in the body of imported HTML format emails will be presented in RefTracker, in the position in which they occurred in the imported email, as a clickable thumbnail, if the image can be downloaded.  If the image cannot be downloaded it will be represented by a clickable anchor tag. Parameter 0.60 provides details of how RefTracker handles images that cannot be downloaded by the email importing process, and provides security options for customers with RefTracker installed locally.

If you are using new question email importing you will need to have parameter 2.41 set to its default setting of "Dialogue only", or above, so that your clients can use the client interface with questions submitted as an imported HTML formatted email.

Setting up email accounts for importing

Each email address that you set up uses a Request form that you specify in the setup, to load the new question into RefTracker. Because different forms can be used for each different email address, emails arriving in the different email addresses can be handled differently according to what you specify in your form design. Allowing a different Request form to be associated with each incoming email address makes this system very powerful. So, for example, reference questions might come into one email address and result in the question being put in the pool. Be careful about setting a form used for email importing to send a confirmation email as this will mean confirmation emails will go to spammers - which is not desirable.
SMS's converted to email could come into another so that their Received via value can be set to SMS and their Priority can be set to high – for SMS's you would most likely not want to send a confirmation to the client but you may want to immediately allocate the question to a staff member to ensure a fast response. You have the full power of Request forms available to you to determine how questions arriving at each email address are handled, and you can have as many different email addresses as you need to allow the arriving emails to be handled in different ways.

To add a new entry to the Email import setup click on the + sign on the top line of the table and complete the boxes presented. When you add a new account description here, an entry will automatically be created for it under System>Batch process menu>Background processing>Email importing. You will need to separately set the parameters for this batch process in order to start the automatic importing.


Note that the setup is different for POP3, IMAP and SMTP drop folders - especially for drop folders!  Be sure to follow the instructions below when completing this table:

Account description: A textual description that identifies the purpose of this entry to you. 

Email Server type: All addresses must be set up as "POP3", "IMAP" or "Mail drop folder". Choose "Mail drop folder" if your IT department has set up a physical folder where mail has been configured to deliver to a physical location by an SMTP server service provided by IIS6.0 (often called "SMTP drop folders").
Your email administrator should be asked to set up this protocol for this email address. If you have an Altarama hosted system, Altarama can provide you with altarama.com, reference-service.info, and libraryresearch.info email addresses, set up with POP3 or IMAP, specifically for this purpose.

Email server: The domain of the email server hosting the email account that will be used for this entry. If you are an Altarama hosted customer you will set this field to pop3reft.altarama.com.
If you are an in-house customer this information will be given to you by your email system manager. 
If they have set up SMTP drop folders you will need to enter the Physical path of the email drop folder e.g. c:\inetpub\mailroot\drop.  UNC file paths to remote devices are also accepted e.g. \\server\inetpub\mailroot\drop\*.eml.  By default the system assumes the mask to be *.eml if a file mask is not specified in the physical path.  Implementation hint: - ensure that the RefTracker application pool user (typically Network Service) has Read/Write/Execute permission on the drop folder, so the mail can be read!
If they set up IMAP, they can use IMAP subfolders.  IMAP subfolders allow for multiple email import processes to be executed against the same email address; for example, two email imports to different request forms or request types could be configured to read from different folders, and a mail rule implemented to filter mail to the appropriate folder.  Alternately, a workflow could be set up whereby mail delivered to a multi-purpose mailbox could be manually sorted by staff in an email client (e.g. triaged, for urgency, or to specific request queues based on message content), and directed (e.g. by drag-and-drop) into the appropriate folder for the request type contained in the message.
An example use for this would be where an organisation is already running one email address for all their library correspondence and they wish to continue to do that.  By using IMAP folders, the email coming into that account can be triaged by a combination of mail rules and human interaction so that questions that need to go into RefTracker are dragged and dropped into subfolders according to their type.  With each subfolder set up as a different import account in RefTracker, they can be brought in using different request forms for each type.  A subfolder can even be set up as the service account, which means that the one email address can be used for both incoming and outgoing email (as long as someone triages email arriving back at that account).
Implementation hint: To accommodate the specification of sub-folder, the syntax for the Email Server field in the Batch Process menu -> Data Import/Export -> Email Importing has changed to the following:
Server:port\folder
  Where:
- Server is the hostname of the server hosting the mailbox, that allows IMAP4 access.
- Optionally followed by a colon and the port number on which to connect to the server (RefTracker defaults to the default IMAP ports if port is not specified).
- Optionally followed by a backslash, and the name of the folder to import from.

Some organisations require that SSL be used when picking up email from a POP3/IMAP account. If you need to connect to a POP3/IMAP sever using SSL include the port number in this field. For example, for Gmail's IMAP service over SSL you would enter imap.gmail.com:993 in this Email server field, and for their POP3 service over SSL you would enter pop.gmail.com:995 in the Email server field. RefTracker's email importing service will automatically detect whether the email server supports SSL and use it if available. Otherwise it will default to trying to connect unencrypted. 

Username: The full email account address that will be used for this entry. All email sent to this account will be picked up and brought into RefTracker.
The POP3 and IMAP protocols used ensure that after an email is picked up by RefTracker it is deleted from this email account so the account must be dedicated to this use with RefTracker.The account used here should be an account that has had spam filtering applied to it to ensure spam questions are not created in RefTracker from spam arriving in this account. In the future Altarama will add an option to apply the same content checking that can be applied to questions coming in through RefTracker Request forms.If you are an Altarama hosted customer you should request email addresses for your service email address and any other email accounts you need for importing from Altarama. They will be in the format xxx@reference-service.info or xxx@libraryresearch.info, or if you subscribe to the Premium secure hosting service xxx@<YourRefTrackerDomain>, where xxx is up to 20 alphanumeric characters.If you are an in-house customer you will need to request creation of these POP3 or IMAP email accounts from your email system manager.
For SMTP drop folders enter the email address for the account you wish to import in this import.  RefTracker will sort the mail in the drop folder and read ONLY messages addressed to this recipient e.g. donotreply@abc.com

Password: This field is not used for SMTP drop folders, BUT must be filled for data validation reasons, so enter any value here when setting up an SMTP drop folder.
For POP3 or IMAP accounts, RefTracker will have to present this password each time it wants to pick up email from this email account. This information will be given to you by whoever set up these emails accounts for you – Altarama for hosted customers, or for in-house customers it will be your email system manager. To edit an email account password, you need to have clicked the edit icon for that email account. A Set password link will then show.



When you click on that link, it gives you the opportunity to enter a new password with the characters blanked out, and enter the password again in the Confirm box. 



To save your new password, click the green tick on that line, as you would for making any other change to this email account definition. If you no longer want to change the password, click the cancel link below the password boxes. 

Import Request form: This is the RefTracker form that will be used to create the new question from questions arriving as emails picked up from this account – so it is only applicable when the Email Account Task is "New Request". The form defines the labels that will be used to display this question and can be used to automatically include other information that is known because of the email address into which the question has arrived such as Request type, Client location, Priority (it will override any automatically detected priority), and Received via.

You can even set up mandatory fields in the form that cannot be completed by the email import, that will result in the form being marked as "incomplete" on import, which will ensure staff are taken to the Change screen to add the incomplete information (for example ensuring Matter numbers are added for questions arriving into legal libraries).

You can use a form that you use for other purposes, or you can design one specifically to suit the emails coming into this email address.  See the "Hints for designing Request forms for use with email importing" section of this manual for important guidelines as to how to design a form for this purpose.  We recommend that the Request form that you use is set up to NOT send a confirmation email, so that confirmation emails are not automatically sent to spammers on the occasions that spam email enters your system (which it is bound to do no matter how good the spam filters in use are!

In new systems a distribution Request form called "Email import" has been provided as an example of the form that would be used for this purpose (if you do not have it you can see it in our Playground system accessible from our supportsite). Note that even though this Request form is set with no client or staff availability, it works with the email import function as email import is a background is a function.

This distribution "Email import form" is also set up use special email templates that provide the client's Subject line in the Subject line of all correspondence re that request.

Dynamic lookups can be used with email importing to have client contact information other than the client's email automatically added to questions created from email imports - see the Dynamic Lookups manual for more details of this if you have the Dynamic Lookups Module. 

Email Account Task: "Service account" defines that the emails coming into this account will all be replies and notifications in relation to the emails being sent out by RefTracker (in other words it's the email address specified in your RefTracker parameter 1.2). "New Request" defines that the emails coming into this account will all be used to create new questions. 

Once an entry has been added to the Email import setup screen you can change it by clicking on the Edit symbol on the far right of its line.

You can delete it by clicking on the Delete symbol on the far right of its line.

Clicking the spyglass icon takes you to a function that allows you to test the setup of this account and see the import logs for that account. There is also a "Schedule email import process" button that allows you to do the import step of setting up the automatic import frequency. See the next section for details of how to use all these functions.

Exercise:

If you follow the instructions above you will have now set up your email account/s for importing. To utilise this setup you now need to set up the process for automatically importing the email arriving into those account using System>Batch process menu>Background processing>Import email, but you can test your setup and link to the batch process setup by clicking the appropriate spyglass "View email log" icon on this page.

View email log functions for an email account set up for importing

When your email account/s are set up you will want to test the setup, and then enable the Background process for them so that emails will be imported automatically. You may also want to view the logs that record the emails imported and processed by RefTracker.

To set up automatic uploading from the email addresses you have defined as enabled, or test your setup manually, and to view logs of pickups from this email account click on the "View batches. . . " spy glass icon on the end of the appropriate account setup line. This is the best place to go when looking for emails that have been imported for that queue (as opposed to the Background processing queue log tab) as it only shows occasions when there was activity, and it show emails imported when the import process is manually run. 

The following screen print shows that the "Service email account" definition associated with the example patch@altarama.com email address (the row with the grey highlight) has imported messages a number of times recently. 


Date entered: the date and time at which the import process was run.
Messages imported:
the number of messages downloaded from the email server. For each of these imported messages, RefTracker will either save it as Pending, Reject it, Add it as a new question, or use it to Update an existing question.
Messages pending:
the number of messages held for possible future re-processing as a new question (System administrators should check whether they truly are a new question, and if click the appropriate "Send to. . . " link in the email to the System administrator, or click on the "View mail log"spy glass icon at the end of the line as the "Send to.  .." function is also available from there).
Messages rejected:
the number of messages for which no action was taken (System administrators should check whether manual action is required in relation to these questions by clicking on the "View mail log" spy glass icon at the end of the line).
Questions added:
the number of new questions created in RefTracker (when the account is a New requests account).
Questions updated:
the number of RefTracker questions updated (when the account is a Service account).

The [Show all messages] link shows a list of all of the email recently received into that account and the current status of each of those emails.

The [Show pending messages] link that appears for the Service email account only, and shows messages that have been imported but not yet processed (because they could not be matched to an existing RefTracker question).  The email address specified by parameter 1.3 will be sent an email whenever a message is put into this Pending status.  This email, and this screen provide [Send to <name> queue] links that allow each pending message to be resubmitted to a New question email account with a single click.  The message you moved will disappear off this Pending list, and be imported by the email importing queue that you choose here, the next time that the email import process runs.
Here's an example of this Pending messages screen:
Messages are automatically cleaned out of this Pending list after the Parameter 0.2 amount of time (usually 1 month), so you do not need to do anything about Pending emails (such as out of office notices, that do not need to be resubmitted (moved to an appropriate new question email import queue).

To set up automatic importing of emails from this account click on the "Schedule email import process" button. Details of how to use this Background processing function are provided in a later section.

To run a manual pickup click on the "Import emails now" button. You might do this in order to test your email setup (a "Failed when processing email account <name>"message indicates that your email account setup is incorrect – an email advising these failure details will be sent to the Active System administrator when this occurs. This ensures that the System Administrator is aware that the service has stopped operating when it is in live usage). The system will bring the details of all emails in that email account into a table of emails and then process each email into RefTracker creating a new question using the Request forms specified if the Mail Account Task is "New question", or creating an update to an existing question where a valid question number can be found in the subject line if the Email Account Task is "Service account". It may take some time to process the emails that have accumulated in your email account (about 1 second per email).  After the upload completes a summary of the run will display below the "Import emails now" button as shown in the screen print above.

Your Active System administrator will be automatically notified via an email, whenever an imported email is rejected or left pending so they can apply the information manually if applicable.


Questions added
indicates a new question was created and the question number created shows in the Question column. You can click on this question number to go directly to the Answer screen for that question.
Questions update
indicates a note was add the Question number indicated in the Question column. An email will have been sent to the staff member currently allocated this question, if any, showing the note that was added and providing links that allow the staff member to work on the question.
Messages pending
indicates an email arriving at the Service email address did not have a valid question number in its subject line and so could not be applied to any existing question. An email will be sent to the System administrator or email address specified in parameter 1.3 containing the details of this question so that it can be handled manually if necessary.   If it happens to be a new question that has arrived at the Service email address the email advising of its arrival will contain a "Send to. . ." link that, with one click, allows this imported email to be represented for processing as a new question.
Messages rejected
indicates an email arriving at the Service email address or a New question address that is not in a valid format for importing  usually spam or an email likely to cause an infinite loop. In the rare case where this is actually a valid update or new question, the System administrator may want to cut and paste the details of this email into RefTracker from this screen or the email they receive about it.

View email log

If you want to view a detailed log of that Email import run you can click on the "click here" link provided in the summary, or, to view the log for any past run click on the spy glass icon at the right hand end of the line of the Account log for that run.Here's an example of a log for a Service Account run.



Each email picked up by the run is listed, whether it was processed or not. Emails that are not processed should be examined by the System administrator and any appropriate manual action should be taken by the System Administrator. The summary shows:
- the four pieces of data that were picked up by the Email import function from each received email: Client name; Email address (the reply to email address if one was supplied by the email), otherwise the from address); Subject; and Body
- Email message status which shows how the message was processed by RefTracker.

A [Show messages for all batches] link shows all emails recently handled by this process - great when you are trying to find a specific email.

The Question link – a link to the question that was created or updated (goes to the Answer screen, Answer tab for newly created questions and to the Details tab of the Summary screen for updates to existing questions). The System Administrator can use this link to work on this question immediately by clicking on it.

What happens when there is a problem with email importing

A number of safeguards and error logs are built into the RefTracker email importing process to ensure that the process operates smoothly:
- RefTracker can handle emails in all known formats, including emails without subject lines, and /or body.
- When an email has more than 20 addresses in an email field, RefTracker logs the error and sends an email to the Parameter 1.3 email address showing the email content and a message about the email address list being too long.  These messages are not automatically imported as they are generally spam messages.
- RefTracker can detect the type of some notices from other email systems, for example non delivery notices can often be detected as to which request they belong, and automatically imported into that request, despite them not having a From address or a Subject line.  Most "Out of office" notices can be correctly ignored.
- Emails that have a From: address that is the same as any of the email addresses being used for importing into your RefTracker system are rejected as being invalid emails as they would cause an infinite loop.
- Emails that have a cc: or bcc: address that is the same as an email address being imported into your RefTracker system are imported but that email address is removed from the cc: or bcc: line as that they would cause an infinite loop of emails.
- The email send function removes any of your RefTracker email importing addresses from the To:, cc:, and bcc: addresses.  This is necessary in order to remove the possibility of infinite loops.  If this results in there being no email addresses to send to, the email will not be sent and the issue is logged to the warn<date>.txt file.
- Where the From: address of an imported email is the same as the To: address, RefTracker attempts to resolve a correct From: address by examining Reply-To:, Return-Path:, and then From: (according to your setting of Parameter 0.13).  This is a common technique used by online databases to allow clients to email requests for articles, to the library.  If another address that is not an email address monitored by this RefTracker system's email importing cannot be determined, the email is not imported, but the System administrator is advised of its content by email.  If this is the case the email is most likely a spam email.

 

If there is a problem with the process of email importing your System administrator, will receive an email detailing the error that is occurring.
If the issue is a "Failure to connect" the System Administrator will need to seek assistance from the manager of the particular email account being monitored in order to resolve these sorts of issues.
The issue may also be related to the RefTracker Request form used for the import process, becoming invalid.  When the error indicates an invalid request form, the form may have become invalid because a code table entry used in that form has been deleted.  Check the Request form and re-save the Default values.  If Dynamic Lookups are in use, check the Lookup code mappings used by that form.

Parameter 0.10 allows you to control the frequency with which these emails are sent when the issue is an email connectivity issue that is continuing to occur.


In order to reduce the impact of email flooding incidents (email importing infinite loops), RefTracker maintains a count of the number of “System error emails” (for both the staff and client interfaces) and “Imported email not processed” emails that are sent and will stop sending these messages (and record an error in the error log instead) once it detects that 15 such emails have been sent in any 15 minute period.  This will result in a maximum of 1440 of these types of emails being sent in any 24 hour period – enough to notice the issue quickly but not enough to flood any email system.  Should more errors than this be generated, the errors will be logged – just not emailed after the first 15 in any 15 minute period. Similarly if more emails cannot be imported, they will still be logged in the email importing log – emails will just not be sent once the limit is reached.

Despite these protections and limitations, email import infinite loops are still possible in instances where there is a problem with the allocated staff member’s email address – their email inbox is full, their email box has been deleted before the allocated staff member is Disabled or Deleted in RefTracker, or the email address is incorrectly entered for a new RefTracker staff member.  It is also possible that an infinite loop could be generated by an email spammer.

Therefore the email importer also keeps a count of the number of “Update by email” notifications sent for any specific question.  If the count exceeds a threshold (default 20 “Update by email” emails sent in relation to any one question within a 2 hour period) then an alert email like the example shown below, is sent to the system administrator.  The notice is also copied to Altarama (cc'd to the To address recorded in the critical.xml file) as it is important that the issue be investigated as soon as possible by clicking the hyperlinked question number in that email – it takes you to the question’s History tab where the frequent updates will be able to be reviewed.  The parameters for this functionality can only be set by Altarama using parameter 80.25.



When a question is imported, but cannot be automatically processed as an update to a RefTracker question ("messages pending") or as a new question (those noted as "Messages rejected"), as appropriate to the queue it arrived in, the System administrator (or another email address that you specify at parameter 1.3) will be sent an email containing the details of the email that could not be imported, so that they can manually address the handling of the email. This ensures that emails sent to RefTracker are never ignored – that they are addressed in some way!

The layout of the email sent will differ according to whether the email came into a Service email account or a New question email account.
Emails rejected by the New question import process need to be handled manually and the email to the System administrator about it clearly describes that.
Use the “View all rejected email. . .” link if you have received a number of these emails – it will show you all emails that have not been able to be processed recently for that email import address.


However emails unable to be imported by the Service email account, may well be a new question inadvertently send to the wrong email address.  For this reason, the email to the System administrator provides a "Send to . . ." function that allows the email to be reprocessed as a new question with just that one click.

The “Send to . . .” links resubmit this email for importing as a new question via the queue that you choose to click (there will be one link for every new question email importing queue defined in your system). When you click a "Sent to. . " link, the system will confirm that the question has been moved to the import queue (so that it can be processed next time that import queue is processed.  If you are logged in, the message will appear at the top of the Home screen (displayed in a separate tab so it does not impact on anything you are currently doing and can simply be closed):

If you are not logged in the message will appear at the top of the login screen.  You do not need to login.  The message will display for 1 minute only - until the login screen next refreshes and the window/tab can be closed at any time:

The next time that queue is processed, the email that failed to import (complete with any attachments) will create a new question using the attributes of the queue that you chose to "Send to. . " it to.  There will be a "Send to. . ." link for each new question email importing process that you have configured in your RefTracker system.

 

If an email arrives at the Service email address without a valid question number in the title, and there is only one open question that uses the email address from which this email came, the email sent to the parameter 1.3 email address will include extra information about the one open request as it might possibly be related.  This is particularly helpful for things like mail box full notices and out of office notices that cannot be automatically handled by RefTracker using other techniques we apply.  Here’s an example of the email that will be sent in this scenario. 

If RefTracker only one question in the system that is from the same email address, a question number hyperlink will be provided that takes you to the Details tab of that question as it might be a match. 

Use the new [Resubmit as update] function if the email is an update to an existing question.
When you click on [Resubmit as an update] you are presented with a display of all the unprocessed emails for the Service email address, with an option to enter the number of the question that you want this email to be applied to as an update.

Enter the Question number in the box provided and click “Check number”.  A summary of that question number will appear.  If it is the right question number, click “Update Question” and the email will be resubmitted for processing as an update to that question number, the next time the Service email address email importing runs

If you need to find the appropriate question number, use the [Search] link to have the Search function open in a separate window/tab.  A search link is also provided in the "Unable to import" email.

 

A "Reject" link is also provided for scenarios where the email in not appropriate to be reprocessed (for example, it is a spam email). When you select the “Reject” option, the incoming email is deleted from the email importing queue.

Use the “View all pending email. . .” link if you have received a number of these emails – it will show you all emails that have not been able to be processed recently, with [Send to. . .]  and [Reject] links allowing them to be reviewed and individually resubmitted from that list. Beware: using [Reject] completely deletes the email from the email import queue.  It cannot be retrieved after [Reject] is used.

 

Don’t forget that you can use “Forward as an attachmentwith $import (or the value set in paramter 0.14) in the subject line, to submit emails that arrive elsewhere (such as in your personal email), to RefTracker.  Forward them to a new question email importing address for them to be imported as a new question, or to your Service email address for them to be imported as an update to a RefTracker question.  If the email that you forward as an attachment to the Service email address, does not have a valid RefTracker question number in its subject line, use the “Resubmit” option, to allow the RefTracker question number that should be updated, to be provided, and the question updated by the next run of the Service email address email importing.

New questions created by email import

Where the Email importing function creates a new question from an imported email, the data that is collected by the Email import function from that email is applied using the Request form specified for this email importing address.

No matter what request form is used to import the email:
- The email address will be placed in the Client email field
- Any cc: email addresses will be placed in the cc: field (and copied on all correspondence in relation to this question)
- If a name can be determined from the email, it will be placed in the Client name field
- The Priority will be matched to a RefTracker question priority using the Priority code table so that a priority indicator can be presented to the left of the Target date in the Open questions screen and any Allocation/Notification email sent to staff can be sent with the same email priority. Note that any Priority built into the email importing request from, will override this system (see the Priority code table for more details).
- The Subject will be placed in the Subject line field.
- The body of the email will be placed in the Question field.  In line Inline images will create images in the Question text.  Inline files that are not images will be converted to standard attachments.  Downloadable images will be represented as thumbnails, if they can be downloaded, and as anchor tags if they cannot be (Parameter 0.60 provides further information about images that cannot be downloaded). 
- Any attachments to the email will become attachments to the RefTracker question.

Here is an example of the log for an email that has been used to create a new question. 

 

Here's how staff see it in the Open questions screen.  Note that the Subject appears on a new line and bolded (just like it does in your email):


When the upload completes, a new question will have been created for each email that is uploaded, and the question will be showing in Open questions, and an allocation email will have been sent if appropriate. This means that at this point the question is already appearing to your staff in the normal course of their answering processes. However, without going to Open questions, a System administrator viewing this screen can click on the question number shown in the log, which takes them to the Answer tab of the Answer screen where they can start working on the question, if appropriate. 



Viewing the Details tab of that screen will confirm that the import process has brought in the information from the email using the labels and fields as defined in the Request form specified for this email account:
- Client email (the "from" email address in the email) is saved into the Client email. The "Email" entry in the "Received via" code table is saved in to the "Received via" field unless another value has been defined in the Request form.The "Email" entry in the "Response method" code table is saved in to the "Response method" field unless another value has been defined in the Request form. cc: (the "cc:" email address/es in the email) is saved into the cc: field.
Client name (the "from" name in the email) is saved into the Client name field, if a name is provided.
- From (the "Reply to" address provided by the email, if there was one, otherwise the "From" address) is saved into the client email address field.
- Subject is saved as the first line of the Question field and is also saved int the "Subject line" field.
- Body is saved as the third and following lines of the Question field.  The second line will always be blank for imported emails.
- Priority will indicate the priority of the imported email unless it has been overridden by a Priority setting built into the Request form(see the Priority code table for more details).

- Any attachments will have been included as question attachments to the newly created question.
- The client Locale field will be set so that follow-up correspondence with the client can be provided in the client's language (if supported by RefTracker) and using the right date formatting.  Locale is determined from the language tags in the header of the email. The following tags are examined to find Locale:
accept-language
acceptlanguage
content-language
If a Locale cannot be determined from any of these tags the Locale will be set to that of your RefTracker system as defined by the email's Language in the System>Language code table.

Where the Request form defined for use with this email account specifies it, 
- the client (and any cc:) email address/es will have been sent a confirmation email.
- the new question will have been allocated to the appropriate staff member, and an allocation email will have been sent to the allocated staff member with the same priority as the imported email; or the question will have been added to the Pool and a Notification email may have been sent to a shared email address with the same priority as the imported email.
- some other fields may have been added as a result of being defined in the Request form, such as Staff location that ensures that these emails are directed to the right staff team/location on arrival. 
- If your system uses Dynamic lookups, a Postpop lookup can be applied during the email import process to add things like phone numbers for clients based on a lookup of your client data on the email address field.
- the question may have been marked as "incomplete" (indicated by a construction icon  near the question number), in which case mandatory information specified by the Request form was not provided by the import process and staff will need to check the question and add the missing information as indicated by a validation message when they work on the question. 

- If the imported email was in HTML format a Question attachment will have been created - the attachment will be called “Source HTML email for <question_no>.htm” where Question number is the number of the RefTracker question created by importing that email.  Click that Question attachment to see the original HTML email, if there is any concern about RefTracker's representation of it.

Viewing the History tab for that question will confirm that all the data in the email was imported into the new question, and will show which email importing address the email came in through.  Clicking on "View email message" shows a copy of the email sent or received, with its header information (showing things like the Priority with which the email was sent).
 

Question updates added by email import

Once a question has been entered, clients can update the question using the links to that functionality in the email they received, but some of your users will use their email reply function to update their questions. If your system has email importing set up, any "reply" email that can be matched to an existing question will be automatically applied to that question as an update. Information about those that can't be matched will be advised to a nominated email address or the System administrator. Without email importing in place these replies will appear in an email address monitored by someone in your organisation.

When an email arrives at the RefTracker service email address, the system will attempt to match it to an existing question. It does this by:- searching the Subject line for a valid RefTracker question number- if none is found it will search the subject line of any emails that might be attached and matches on any valid RefTracker question number found there. This allows automatic matching of many types of non-delivery notices.- if a match is still unable to be made the system will search all open questions, and if only one open has a client email address that matches the email being processed it will be matched to that question. This allows for automatic matching of emails with simple email address typing errors.- if a match is still unable to be made, the contents of the email that can't be matched, and details of the sender, will be emailed to any email address specified in parameter 1.3, or if none is specified, to the System Administrator.

Once a match between arriving email and an existing question has been made, it is handled as follows:

If the email is from the client – where the client is defined as any email address in the client_email field (but not from any cc:, or bcc: fields):

    • if the question is closed then the question is reopened with status "Re-opened by client(imported)" and a History entry is created, the client receives a reopen confirmation email and the allocated staff member is also advised of the update.
    • if the question is open then an update note is added to the History, the status goes to "Updated by client", the client receives a confirmation of update, and the allocated staff member is advised by email of its arrival. 

If the email is from someone in a cc: or bcc: field:

    • if the question is closed then the question is reopened with status "Re-opened by copied party" and a History entry is created, and the allocated staff member is also advised by email of the update.
    • if the question is open then an update note is added to the History, the status goes to "Updated by copied party", and the allocated staff member is advised by email of its arrival.

If the email is from the allocated staff member, we can presume that the staff member knows they are updating the question and so they do not need to be advised:

    • if the question is closed, the question is re-opened with the status "Re-opened (imported)" and that is noted in the History.
    • if the question is open, an update note is added to the History, the status of the question is not changed. 

If the email is from any other staff member:

    • if the question is closed, the question is re-opened with the status "Re-opened by staff (imported)" and that is noted in the History, and the allocated staff member is advised by email of its arrival.
    • if the question is open, an update note is added to the History, and the status of the question goes to "Updated by staff", and the allocated staff member is advised by email of its arrival. 

If the email is from anyone else:

    • if the question is closed, the status is set to "Re-opened by third party (imported)" and noted in the History, a confirmation email is not sent to the sender, but an email advising of the update is sent to the allocated staff member.
    • if the question is open, the status is set to "Updated by third party" and noted in the History, a confirmation email in not sent to the sender, but a notification of the update is sent to the allocated staff member.

If the allocated staff member is Available, they will receive an email advising them of the update/reopen created by this imported email.
If the allocated staff member is Unavailable, the question will be reallocated using the following escalation routine:

  • the allocated staff member's Work reviewer if they have been involved with this question, and if that Work reviewer is Available, or, if the previously involved Work reviewer is Unavailable or there has been no previously involved Work reviewer,
  • the currently logged on and Available Supervisor at the allocated staff member's Location with the lowest number of open questions, or, if none,
  • the Available Supervisor at their Location with the lowest number of open questions irrespective of whether they are currently logged on, or, if none,
  • the currently logged on and Available System administrator at their location with the lowest number of open questions, or, if none,
  • the Available System administrator at their location with the lowest number of open questions irrespective of whether they are currently logged on, or, if none,
  • the Active System administrator even if they are Unavailable.


Here is an example of the email import log for an email that has been used to create an update.



When the upload completes, an update has been added to the Log of the appropriate question, for each email that had a valid RefTracker question number in the Subject line, and the Allocated staff member, if any, has been advised of the arrival of the update via an email. The History entry that is created will be indicated as an "Email response" type entry and will consist of:

Client name: Client name obtained from the email (if any, otherwise "Client")
Subject:
followed by the subject line from the email
Body:
body from the email.

Also the System administrator we can click on the question number in the log and will be taken to the Details tab of the Summary screen for that question. In that screen you will see the note that was added to the question. If the System Administrator wants to work on the question they can click on Answer. In the Answer screen, the note just added will appear at the top of the History tab.



And if we look at the Correspondence tab, here is what has been recorded. You can see the entry for the Note, and for the email that was sent to the staff member notifying them that the note had been added. There is no confirmation email to the client as a confirmation email in this scenario would provide no useful information, and perhaps even be confusing.And similarly the full details of the note have been added to the History tab, where you can see that the Status of the question have been appropriately changed to "Updated by client".



And here is an example of the notification email that is sent to the staff member. Links that allow the staff member to go directly to the Answer, Details, History or Reallocate screen are provided in the email to make it easy for staff to provide the appropriate action, if any, to this update.



Exercise

When you have finished examining the Email importing function and setting up the accounts that you will need for your organisation, go to System>Batch process menu>Data export/import>Time and cost Journal entry import.

...

Zen Master
Administration Training Manual
titleyes
labelsyes
other-master-page-titleAdministration Training Manual
use-other-page-as-masteryesother-master-page-title
commentsyes
masterown-master
commentsbreadcrumbsyes
metadatayes
breadcrumbsyes