Batch processing allows you to schedule file transfers to take place automatically. Once batch processing is set up, you do not need to be present for the file transfer to take place. This capability requires additional infrastructure and configuration work.
When using the Batch Processor, the PostalOne! servers return file transfer/upload status feedback to the sender’s workstation automatically. The feedback information (receipt file) is written to the sender’s workstation in either a delimited ASCII text or XML format file. You can load the receipt file into a spreadsheet or a database for storage and viewing. A receipt file is saved in the same directory that contains your mailing jobs. For a detailed explanation of the contents of a receipt file and associated record types and format, see Appendix C. Status Receipt File Layout and Appendix D. Postage Statement Receipt File Layout. Status receipt files are currently the only supported receipt file.
Prerequisites for Batch Processing Hardware
For batch file transfers using UNIX, the minimum workspace requirements are a workstation with a 1 GHz processor and 256 MB RAM. For batch file transfers using Windows XP, the minimum workstation requirements are a Personal Computer (PC) with a Pentium IV 1 GHz processor and 256 MB RAM. The exact RAM requirements vary depending on the size of the Mail.dat job. Table 4-1 details the RAM requirements.
NOTE: The Batch Processor can also be run on machines with HPUX operating systems.
Table 4-1. RAM Requirements Based on Net Job Size
Net Size of Mail.dat Job (MB) *
|
Recommended Physical RAM
| |
0 – 10
|
64 MB
|
11 – 25
|
128 MB
|
26 – 50
|
256 MB
|
>50
|
>1024 MB
|
* The net size of a Mail.dat job is calculated as the "total size of all files" associated with a job, minus the size of the PQT, SNR, and PDR files.
For Mail.dat jobs with a large total file size, file transmission may result in an “Out of Memory” error. To resolve this issue, modify the JVM memory arguments, then retry the transmission. For more information on how to modify the memory settings for batch processing, see Section 3.1.8.
NOTE: These memory requirements are for a single thread; when sending multiple simultaneous jobs, use the sum of all net sizes to estimate memory requirements.
Software
The following software is required to use the Batch Processing functionality:
Operating System – Microsoft Windows XP for PCs or Sun Solaris 2.6 for UNIX Workstations.
Java Software – Sun Microsystems Java Runtime Environment (JRE) or SDK, latest revision of Version 1.4.2 or greater.
This is available from Sun Microsystems, Inc. at http://java.sun.com/products/archive/index.html. Select Java 2 JRE or SDK, Version 1.4.2 or greater. Review and accept the download terms and conditions. Next, select the JRE or SDK that corresponds to your operating system configuration. Then select a “Typical” installation, and note the directory to which the Java software is installed (e.g., C:\program files\java\j2re1.4.1\bin).
Scheduling software – software that lets you schedule batch processing of your mailing files to run on the dates and at the times you require. The default scheduling utility for the operating system on the machine you use to send mailing files (e.g., cron/UNIX or “AT” task scheduler/Windows XP) may work.
Digital Certificate – get and install a digital certificate for each server that is used to send mail files. For more information, see Section 3.1.6, Digital Certificates/Security.
Network
The Batch Processor software uses the http/https protocol to communicate through port 444. If firewall settings prevent http/https communication through port 444, reconfigure the firewall to allow this traffic.
Downloading and Configuring the Windows Batch Processor
To download and configure the Windows Batch Processor:
Launch Windows Explorer and in your root directory, create a PostalOne! Batch Processor installation directory. This is where you will place the batch processor software.
NOTE: Give the folder a name that you will readily remember, such as: C:\MAILDAT.
Log on to the PostalOne! system, http://www.uspspostalone.com/.
On the left menu bar, click Download Batch Processor and download the following files into the installation directory:
WinZip file – Java archives that perform the batch processing
Parameters Files – Local host parameter for the batch processor
Extract the files from daemon.zip, using WinZip, into the PostalOne! Batch Processor installation directory (double-click the zip file and extract the contents). Once extracted, the directory should contain:
daemon.jar – Java archive file
jsse.jar – Java Secure Socket Extension (JSSE) Java archive file
jcert.jar – JSSE Java archive file
jnet.jar – JSSE Java archive file
xercesImpl.jar – JSSE Java archive file
client.config – Client configuration file that specifies parameters to connect to the PostalOne! system
postal1.ini – Daemon initialization file that specifies local host parameters
Copy the three JSSE archive files to the extension library ( …\lib\ext ) directory created when the Java JRE / SDK software was downloaded and installed:
For example:
Java Runtime Environment - C:\Program Files\JavaSoft\JRE\1.4\lib\ext
Java 2 SDK - C:\jdk1.4\jre\lib\ext
Create a “Repository” subfolder in the PostalOne! batch processor installation directory.
This folder is a repository for your Mail.dat files. For example: C:\MAILDAT\Repository.
Create two subfolders in the new repository folder—one for successful transmissions and one for unsuccessful transmissions. For example: C:\MAILDAT\Repository\Successful and C:\MAILDAT\Repository\Unsuccessful. The batch processor deposits transmission files in these folders as each job is processed.
Edit the hostName parameter of the client.config file to connect to the desired server. Use your preferred text editor, such as Notepad, to edit the client.config file.
To connect to the Customer Acceptance Test (CAT) environment, set:
hostName = upload9.uspspostalone.com
To connect to the Production environment, set:
hostName = upload1.uspspostalone.com
The default hostName connects to the Production environment.
After you have created all three subfolders, edit the Postal1.ini file using Notepad, for example, and make the following modifications to the [Repository] section to reflect the directory structure you created above.
[Repository]
RepositoryLocation=C:\MAILDAT\REPOSITORY
SuccessfulRepositoryLocation= C:\MAILDAT\REPOSITORY\SUCCESSFUL
UnSuccessfulRepositoryLocation= C:\MAILDAT\REPOSITORY\UNSUCCESSFUL
NOTE: Postal1.ini is located in your batch processor installation directory (i.e. C:\MAILDAT).
The parameters required to efficiently use the PostalOne! system are provided in the table below.
Table 4-2. Postal1.ini Parameters for Windows
Parameter Name
|
Valid Values
|
Default
|
Description
| |
MaximumOpenThreads
|
1-50
|
4
|
Maximum threads that can be open at one time.
|
RepositoryLocation
|
Any valid directory path.
|
C:\postal1\data
|
Location of Mail.dat files waiting to be processed.
|
SuccessfulRepositoryLocation
|
Any valid directory path.
|
C:\postal1
\SUCCESSFUL
|
Location where the Mail.dat files are moved after they are successfully processed.
|
UnSuccessfulRepositoryLocation
|
Any valid directory path.
|
C:\postal1
\UNSUCCESSFUL
|
Location where the Mail.dat files are moved if the processing fails.
|
ReceiptFileRepositoryLocation
|
Any valid directory path.
|
C:\postal1
\ReceiptFile
|
Location where the receipt files are written by the application. This is the default for both status and postage statement receipts. However, status receipts use it only if PSReceiptFileRepositoryLocation is set.
|
ReceiptFileName
|
Any valid file name.
|
Report
|
The name of the receipt files. Note that a unique three-digit sequence number is used to define the extension of the receipt files.
|
Delimiter
|
Any printable ASCII character.
|
,
|
This character delimits all receipt file fields.
|
NumberOfValidationLogFiles
|
Any Numeric value. *
|
5
|
Number of validation log files that are written by the application before the first file is overwritten.
|
NumberOfServerLogFiles
|
Any Numeric value. *
|
5
|
Number of server log files that are written by the application before the first file is overwritten.
|
MaximumValidationLogLength
|
Any Numeric value. *
|
100000
|
The maximum size (in bytes) of a single validation log file.
|
MaximumServerLogLength
|
Any Numeric value. *
|
1000000
|
The maximum size (in bytes) of a single server log file.
|
SystemUsername
|
A valid PostalOne! account User Name.
|
Jdoe
|
A valid PostalOne! user name belonging to an active account that allows the account holder to transfer files.
|
SystemPassword
|
A valid PostalOne! account password.
|
Password
|
A valid PostalOne! password belonging to an active account that allows the account holder to transfer files.
|
FileType
|
MAILDAT or DELCON
|
MAILDAT
|
Specifies the type of file that is processed by the application. MAILDAT for transferring Mail.dat files, and DELCON for transferring e-VS manifest files.
|
StatusReceiptFilename
|
Any valid file name.
|
receipt
|
Determines the name of status receipt file. The PostalOne! system appends a three-digit number to the name, which is incremented with each new receipt.
|
PostageStatementReceiptFilename
|
Any valid file name.
|
psreceipt
|
Determines the name of the postage statement receipt file. The PostalOne! system appends a three-digit number to the name, which is incremented with each new receipt.
|
PSReceiptFileRepositoryLocation
|
Any valid directory path.
|
|
Location where the postage statement receipt files (if set) are written by the application.
|
PSReceiptType
|
XML or DEL
|
|
Determines the format of the postage statement receipt file, either XML or character delimited (DEL).
|
StatusReceiptType
|
XML or DEL
|
|
Determines the format of the status receipt file, either XML or character delimited (DEL).
|
* NOTE: Sufficient disk space should be available to store the number of specified files.
The parameter MaximumSystemErrors is no longer required in the postal1.ini file. If it is still there, you should remove it.
Downloading and Configuring the UNIX Batch Processor
To download and configure the UNIX Batch Processor:
Create a PostalOne! Batch Processor installation directory. The batch processor will be located in this directory. For example: /postal1/.
Log on to the PostalOne! system, http://www.uspspostalone.com/.
On the left menu bar, click Download Batch Processor and download the required files to your batch processor installation directory:
Archive file (daemon.tar) – Java archives that perform the batch processing
Parameter File (postal1.ini) – Local host parameters for the batch processor
Use tar to extract the files into the installation directory. Once extracted, the directory should contain:
daemon.jar – Java archive file
jsse.jar – Java Secure Socket Extension (JSSE) Java archive file
jcert.jar – JSSE Java archive file
jnet.jar – JSSE Java archive file
xercesImpl.jar – JSSE Java archive file
client.config – Client configuration file that specifies parameters for connecting to the PostalOne! system
postal1.ini – Daemon initialization file that specifies local host parameters
Copy the three JSSE archive files to the extension library directory ( …\lib\ext ) created when the Java software was downloaded and installed.
For example:
Java Runtime environment - j2re1_4_2/lib/ext
Java 2 SDK - j2se/jre/lib/ext
Once the files are successfully copied or downloaded, create a “repository” subfolder in the installation directory. This folder is a repository for your Mail.dat files, for example: /postal1/data.
Create additional two subfolders—one for successful transmissions and one for unsuccessful transmissions. The batch processor deposits transmission files in these folders as each job is processed, for example: /postal1/successful and /postal1/unsuccessful.
Edit the hostName parameter of the client.config file to connect to the desired server. Use your preferred text editor, such as Notepad, to edit the client.config file.
To connect to the Customer Acceptance Test (CAT) environment, set:
hostName = upload9.uspspostalone.com
To connect to the Production environment, set:
hostName = upload1.uspspostalone.com
The default hostName connects to the Production environment.
After you have created all three subfolders, edit the postal1.ini file and make the following modifications to the [Repository] section to reflect the directories you created:
[Repository]
RepositoryLocation=/postal1/data
SuccessfulRepositoryLocation=/postal1/successful
UnSuccessfulRepositoryLocation=/postal1/unsuccessful
NOTE: For UNIX servers, file paths in the postal1.ini file must be all lowercase.
The postal1.ini file is located in your initial installation directory. The parameters required to efficiently use the PostalOne! system are provided in the table below.
Table 4-3. Postal1.ini Parameters for UNIX
Parameter Name
|
Valid Values
|
Default
|
Description
| |
MaximumOpenThreads
|
1-50
|
4
|
Maximum threads that can be open at one time.
|
RepositoryLocation
|
Any valid directory path.
|
C:\postal1\data
|
Location of Mail.dat files waiting to be processed.
|
SuccessfulRepositoryLocation
|
Any valid directory path.
|
C:\postal1
\successful
|
Location where the Mail.dat files are moved after they are successfully processed.
|
UnSuccessfulRepositoryLocation
|
Any valid directory path.
|
C:\postal1
\unsuccessful
|
Location where the Mail.dat files are moved if the processing fails.
|
ReceiptFileRepositoryLocation
|
Any valid directory path.
|
C:\postal1
\receiptfile
|
Location where the receipt files are written by the application.
|
ReceiptFileName
|
Any valid file name.
|
report
|
The name of the receipt files. Note that a unique three-digit sequence number is used to define the extension of the receipt files.
|
Delimiter
|
Any printable ASCII character.
|
,
|
This character delimits all receipt file fields.
|
NumberOfValidationLogFiles
|
Any Numeric value. *
|
5
|
Number of validation log files that are written by the application before the first file is overwritten.
|
NumberOfServerLogFiles
|
Any Numeric value. *
|
5
|
Number of server log files that are written by the application before the first file is overwritten.
|
MaximumValidationLogLength
|
Any Numeric value. *
|
100000
|
The maximum size (in bytes) of a single validation log file.
|
MaximumServerLogLength
|
Any Numeric value. *
|
1000000
|
The maximum size (in bytes) of a single server log file.
|
SystemUsername
|
A valid PostalOne! account User Name.
|
Jdoe
|
A valid PostalOne! user name belonging to an active account that allows the account holder to transfer files.
|
SystemPassword
|
A valid PostalOne! account password.
|
password
|
A valid PostalOne! password belonging to an active account that allows the account holder to transfer files.
|
FileType
|
MAILDAT or DELCON
|
MAILDAT
|
Specifies the type of file that is processed by the application. MAILDAT for transferring Mail.dat files, and DELCON for transferring e-VS manifest files.
|
StatusReceiptFilename
|
Any valid file name.
|
receipt
|
Determines the name of the status receipt file. The PostalOne! system appends a three-digit number to the name, which is incremented with each new receipt.
|
PostageStatementReceiptFilename
|
Any valid file name.
|
psreceipt
|
Determines the name of the postage statement receipt file. The PostalOne! system appends a three-digit number to the name, which is incremented with each new receipt.
|
PSReceiptFileRepositoryLocation
|
Any valid directory path.
|
|
Location where the postage statement receipt files (if set) are written by the application.
|
PSReceiptType
|
XML or DEL
|
|
Determines the format of the postage statement receipt file, either XML or character delimited (DEL).
|
StatusReceiptType
|
XML or DEL
|
|
Determines the format of the status receipt file, either XML or character delimited (DEL).
|
* NOTE: Sufficient disk space should be available to store the number of specified files.
The parameter MaximumSystemErrors is no longer required in the postal1.ini file. If it is still there, you should remove it.
Digital Certificates/Security
The PostalOne! system uses the Secure Sockets Layer (SSL) version 3.0 to transfer files securely over the Internet. SSL is a secure enhancement to the standard Transmission Control Protocol/Internet Protocol (TCP/IP). It uses a combination of cryptographic processes to authenticate the host computers and to encrypt and decrypt data transferred between them. To transfer files securely (Batch only) with the PostalOne! system, complete the following tasks to permit SSL to operate on your host system:
Create a Keystore
Create a Certificate Signing Request
Submit your CSR and procure a Digital Certificate
Add the Signed Certificate to your Keystore
Enable Encryption on the Batch Processor Client
Test your Signed Certificate
Test the Batch Processor Setup
Each of these steps is discussed in detail below.
Creating a Keystore
A keystore is a file that contains a digitally signed certificate. The Batch Processor file transfer system uses this certificate to verify the name of your host computer before accepting files from it.
To create a keystore, use the keytool utility that is part of the Java JRE/SDK software you downloaded earlier. You will need to know the following information to create a keystore:
The fully-qualified domain name of your file transfer computer (i.e., ARLNVAUX002.usps.gov)
NOTE: The common name must match the domain name.
The name of your organizational unit (i.e., Marketing)
The name of your organization (i.e., United States Postal Service)
NOTE: The formal name must be alphanumeric characters (no @ or &, for example).
The city, state or province, and two-character country code of your location.
An example of creating a keystore follows:
C:\(directory of keytool Java software utility)> keytool -genkey -alias postalone -keystore p1keys -keyalg rsa
NOTE: You must specify -keyalg rsa when creating a keystore for use with the PostalOne! system. The PostalOne! system uses the RSA signature algorithm for signing certificates.
Enter keystore password: postal1 (for example)
What is your first and last name?
[Unknown]: ARLNVAUX002.usps.gov
What is the name of your organizational unit?
[Unknown]: Marketing
What is the name of your organization?
[Unknown]: United States Postal Service
What is the name of your city or locality?
[Unknown]: Arlington
What is the name of your state or province?
[Unknown]: Virginia
What is the two-letter country code for this unit?
[Unknown]: US
Is correct?[no]:
[Unknown]: y
Enter key password for
(RETURN if same as keystore password):
NOTE: Record your keystore password in a safe place. You will need it in subsequent steps.
Creating Certificate Signing Requests
Use keytool to create your certificate signing request (CSR).
NOTE: Use the directory where the keytool software utility resides.
C:\(directory of keytool java software)>Keytool –certreq –alias postalone – keystore p1keys –file p1req.txt
Enter keystore password: postal1 (for example)
Your CSR is contained in the file p1req.txt.
Submitting Your Certificate Signing Request
You must now submit your CSR for digital signing, i.e., procure a digital certificate. Currently, the PostalOne! system uses digital certificates issued by VeriSign. To obtain a certificate, go to http://www.verisign.com/products/site, and obtain a basic 40-bit SSL Secure Server ID for each machine that will send batch files to the PostalOne! system.
NOTE: Your digital certificate generally expires after one (1) year. You need to renew it after that.
After you submit a CSR to VeriSign, you will receive a digitally signed certificate. You must then add this signed certificate to your keystore. The file transfer system uses this signed certificate to verify the identity of your host computer before accepting any files from it.
Begin by getting the reply certificate from VeriSign and copying it into a text editor. It should look similar to the following example:
-----BEGIN CERTIFICATE-----
MIICdjCCAiACEBKVRKjE/nsXwEii3fN6k9AwDQYJKoZIhvcNAQEEBQAwgakxFjAU BgNVBAoTDVZlcmlTaWduLCBJbmMxRzBFBgNVBAsTPnd3dy52ZXJpc2lnbi5jb20v cmVwb3NpdG9yeS9UZXN0Q1BTIEluY29ycC4gQnkgUmVmLiBMaWFiLiBMVEQuMUYw RAYDVQQLEz1Gb3IgVmVyaVNpZ24gYXV0aG9yaXplZCB0ZXN0aW5nIG9ubHkuIE5v IGFzc3VyYW5jZXMgKEMpVlMxOTk3MB4XDTAyMDMwNDAwMDAwMFoXDTAyMDMxODIz NTk1OVowgY4xCzAJBgNVBAYTAlVTMREwDwYDVQQIEwhWaXJnaW5pYTESMBAGA1UE BxQJQXJsaW5ndG9uMSUwIwYDVQQKFBxVbml0ZWQgU3RhdGVzIFBvc3RhbCBTZXJ2 aWNlMRIwEAYDVQQLFAlQb3N0YWxPbmUxHTAbBgNVBAMUFEFSTE5WQVdaMTg0LlVT UFMuR09WMIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDGzthcIrSzSHFHl3x1 ojzhPjacRUSH1vAuk8SZN1VVRrwi76dl4VqtsGpMD/YjmmMbN1fwgTFm6rfpmRXF rWbQaMe9aB6EPZPwOhpEYdTY3Z9QpOwyKdu6amCwyQzUKkNV7sOeycSe+e9cW4JK 0Koxey14YvLkEu03DHGi0zQ0BwIDAQABMA0GCSqGSIb3DQEBBAUAA0EAv5bA+oCG AzzhZsMiclS2O3liuVN3ppLrRU+DdCvi6otcpC0xUYVYjXFi83POzgjmdr0RJt2g
QSWT9fCoL28lgA==
-----END CERTIFICATE-----
Ensure the certificate context (the text between the BEGIN CERTIFICATE line and the END CERTIFICATE line) has no spaces or return characters when copying and pasting. If there are any spaces or return characters, delete them. Put one and only one new line (i.e. carriage return) without any space after the END CERTIFICATE line. Then save the file (e.g., p1cert.txt).
After you have saved the signed certificate, use keytool to import your certificate into your keystore.
C:\(directory of keytool java software utility)>keytool -import -trustcacerts -file p1cert.txt -alias
postalone -keystore p1keys
Enter keystore password: postal1 (for example)
Certificate was added to keystore
Now use keytool to verify that your signed certificate is in your keystore:
C:\postalone>keytool -list -keystore p1keys -alias postalone -v
Enter keystore password: postal1 (for example)
Alias name: postalone
Creation date: Wed Oct 29 11:41:01 EDT 2003
Entry type: trustedCertEntry
Owner: CN=ARLNVAUX002.usps.gov, OU= Marketing, O=United States Postal Service, L=Arlington, OID.2.5.4.17=94497, ST=VA, C=US
Issuer: CN=USPS CA, OU=www.usps.com/CPS, O=United States Postal Service, C=US
Serial number: 2895
Valid from: Tue Oct 28 14:08:07 EDT 2003 until: Wed Oct 27 14:08:07 EDT 2004
Certificate fingerprints:
MD5: 69:2C:04:E3:B7:F1:A9:1F:DE:FD:0E:36:A2:1B:FF:12
SHA1:E5:91:36:25:76:2B:CA:E0:DF:DD:3C:2B:91:E8:3B:97:B3:E6:8D:CE
NOTE: In the example, the certificate is valid for one year, then it expires.
Enabling Encryption on the Batch Processor Client
To enable the batch processor client secure file transfer feature, you must copy your keystore file to the directory in which you have installed the batch processor client software. You must also edit the client.config file. The client.config is already located in the directory in which you have previously installed the batch processor client software.
After copying your keystore file (for example, p1keys) to the directory containing the batch processor client, edit the client.config file. To enable encryption, you must set the encryption parameter to true. You must also supply the name of your keystore file and its corresponding password. When you are finished, these entries should look similar to the following example:
encryption = true
keystore = p1keys
password = postal1 (for example)
Testing Your Signed Certificate
Your batch processor client software gives you the ability to test the installation of your certificate. This test is performed from the command prompt on your local file transfer host.
To test the installation of your signed certificate, make sure the encryption parameter of your client.config file is set to true. Then execute the following command.
NOTE: Use the batch processor installation directory.
C:\(batch processor installation directory)>java –cp Daemon.jar Daemon -testlocal
If your certificate is valid and is installed properly, you will see the following:
Local Server Listening for Client Connection
Response Code is : 200
Response Message is : OK
This (Certificated) Local Connection test is SUCCESSFULL!
Testing Batch Processor Setup
The batch processor client software also lets you test that the batch processor has been set up correctly and can communicate with the PostalOne! system. This test is run from the command prompt on your local file transfer host.
NOTE: Use the batch processor installation directory.
C:\(batch processor installation directory)>java –cp Daemon.jar Daemon -testremote
Scheduling Batch Jobs (Running the Batch Processor)
To have your jobs run automatically, use a scheduling utility/program that allows you to set the date and time the batch file transfer processor runs. Copy or move the mailing files into the repository directory before the batch job executes. To run the batch processor application, you invoke it at the command prompt. For example:
java –cp C:\ \daemon.jar Daemon
Where is the directory containing the Java archive file (daemon.jar), the Daemon initialization file (postal1.ini), and client configuration file (client.config).
Adjusting Memory Limits for Batch Transfers
NOTE: It is only necessary to adjust memory limits if attempts to transfer large files result in an “Out of Memory” error. Do not increase memory size unless necessary.
When the users upload Mail.dat jobs with large file sizes, the computer sending the files (the client) may run out of memory, causing the upload process to fail with an “Out of Memory” error.
To resolve the issue, first verify the amount of memory available on the client computer is sufficient. Next, edit the client.config file to use a larger value for maximumHeapSize, such as 512 (512 MB). The client.config file is located in the batch processor installation directory. For example:
maximumHeapSize=512
Save and exit the client.config file, then retry the transfer. If the Out of Memory error reoccurs, increase maximumHeapSize to a higher number. For example, set maximumHeapSize=768 to use 768 MB, then retry the upload process.
NOTE: Ensure that any memory limit entered is less than the amount of physical memory (RAM) on the client machine.
|
