Read ACORD Standards Blank Document Template text version

TCF SUPPORT FOR LARGE FILES: DRIVING IMPLEMENTATION ACROSS THE INDUSTRY

Introduction

o With the increasing volume of ACORD-based messages, many ACORD members need to exchange very large data such as catastrophe risk models, claim videos and other underlying data. Several ACORD members have approached ACORD asking for assistance with transferring very large files using ACORD DRI Standards. ACORD published a reference implementation guide for this in December last year, after consulting the ACORD Web Services Working Group. The solution was considered by all interested parties, including major gateway providers in the UK & US, as well Microsoft, Sun and IBM. It was put to a formal ballot and approved unanimously. The ACORD Testing & Certification Facility (TCF) has now been updated to support the solution. This allows ACORD to not only certify vendors as able to transfer large files, but also allowed ACORD to: o o o assist vendors by providing a testing facility for them to use during development demonstrate the solution's feasibility and low time-to-market development cost provide metrics to enable potential implementors to model ROI and associated benefits.

o

o

o

Results

ACORD has tested the solution with both .NET and Java based solution providers, and can confirm it is feasible and interoperable. The solution does not require an upgrade to a new version of the standard ­ it is backward-compatible with all versions of DRI & AMS. Key statistics are: DRI o Documents of up to 1.3Gigabytes have been regularly transferred between the US (East Coast) and Germany at an average speed of 500kb/second. o Network utilization of ~0.5% meant that other message could and were exchanged during the download of large files. o Server memory and CPU usage remained unchanged during and immediately prior to a large file transfer; no measurable load was caused to the server, whether sending or receiving. Development o Development took 10 man days, including testing and deployment o Code used generic standard .NET framework classes which are widely used and understood. o The actual code used has been donated to ACORD and is available on request.

Next Steps

ACORD wishes to evangelise this solution and ensure that all parties who may wish to implement are both able to build large file support into their, or their solution provider's, release schedule, and aware of the support available from ACORD

© 2008 ACORD CORPORATION ­ ALL RIGHTS RESERVED.

1

Technical Analysis of Problem Domain

Current practice when transferring documents is to send a DRI message, with all referenced files supplied "instream" as part of the same message. This requires that the files are "encoded" to ensure safe transit inside the SOAP message, and it has been found that it is impractical to transmit files larger than roughly 20Mb using this method. This is because the encoding and decoding processes both require a contiguous working memory set approximately 400% larger than the original file size, as well as substantial processing power. Many servers are therefore unable to encode or decode large files without hitting memory or CPU constraints, which introduces an unacceptable risk of unexpected transaction failures. In Q4 last year, and with reference to industry partners, and taking note of best-practice outside the insurance vertical, ACORD developed technical recommendations to enable the transfer large files using the existing AMS capabilities. The solution involves: a) Receiving ACORD DRI messages using AMS which contain references to large files b) Parsing the AMS envelope and identify the location of the referenced large file(s) c) Downloading these files automatically and making them available to the DRI receiving agent in a way similar to the way "in stream" documents are passed on. Outgoing messages have been similarly configured so that the files can either be contained in-stream with the message or as a URL - or potentially as a combination of the two, if multiple files are being transferred.

Summary of Solution Development

Initially, a simple project was created to try to connect to a web server and download a document, using the standard .NET Framework Classes, HttpWebRequest and HttpWebResponse. During the initial testing of the HTTP download, the complete response stream was read before trying to write it to the file system. This caused a memory error after approximately 250MB. The code was then modified to read the response stream in blocks of one kilobyte, with each block being written to the file system before reading the next block. This removed the memory issues, as no more than one block was buffered in memory at any one time. This allowed files of much greater size to be downloaded without any significant impact on the server resources. The file sizes tried were between 432Mb and 1.3GB. If the file could not be down loaded, the HTTPWebResponse stream automatically returned the error message, such as a HTTP 404 error (if the file could not be found, for example). Once it had been established that it was possible to download files of this size the code was then incorporated into the TCF. Incoming message SOAP envelopes are checked, and if the <ac:CommunicationChannelCd> element contains `in_stream' then the message is processed in the standard way, with the message being validated, the attachments downloaded and then a synchronous PostRs/SOAP Fault being issued. If, however the <ac:CommunicationChannelCd> element contains `url_address' then the message is validated as before but the synchronous PostRs/SOAP Fault is issued BEFORE the attachment is downloaded. The attachments are then downloaded before the DRI Response message is generated, if the attachment downloads correctly then a DRI Response is sent with the <AcknowledgementStatusCd> element being set to `acknowledged' assuming all other parts of the message to be correct. If for some reason the attachment does not download correctly then the <AcknowledgementStatusCd> element is set to reject and the reason for the failure given in the <ResponseDescription> element (i.e. `nofile.txt The remote server returned an error: (404) Not Found.').

© 2008 ACORD CORPORATION ­ ALL RIGHTS RESERVED.

2

Information

ACORD Standards Blank Document Template

2 pages

Find more like this

Report File (DMCA)

Our content is added by our users. We aim to remove reported files within 1 working day. Please use this link to notify us:

Report this file as copyright or inappropriate

385168


You might also be interested in

BETA
ACORD Standards Blank Document Template
ContentMaster_DS_v4.qxp
IBM Software Supporting ACORD Insurance Standards