how to use a proxy service for transferring files

bhaskar_annamalai.trimble.com's picture
We would like to use the wso2 ESB proxy service to transfer a file from one location to another using vfs transport listener. The file is actually a report generated by our server. The proxy service needs to poll the location for any report file and put it in another sftp url. We dont have any service running which ESB proxy service can call in its insequence. It just needs to upload the file from one location to another sftp url. Please let me know the way to achieve this.
indika's picture

 Hi  I believe you have

 Hi  I believe you have looked  VFS sample[1]. Can't use it for your scenario?...  to adopt it ... You never want a service EPR ... You can use a VFS URI as the endpoint of the proxy service      <endpoint> <address uri="vfs:file:///home/user/test/out"/> <!--CHANGE--> Your file location </endpoint>   And you can remove the publishWSDL attribute from the proxy service if there is no act serivce  Thanks Indika  [1] http://wso2.org/project/esb/java/2.1.1/docs/samples/transport_samples.html#Sample254    
indika's picture

 Hi  I believe you have

.
bhaskar_annamalai.trimble.com's picture

Hi, I tried something

Hi, I tried something similar to the sample. the following is the synapse.xml entry. 15 MOVE vfs:sftp://:@:22/home/ file://C:/testvfs/in .*.txt text/plain this opens a vfs listener in a sftp path for .txt file and finally moves it to file://C:/testvfs/in. But, i get the following error when some .txt file is put in the sftp url. [2009-11-11 09:59:19,463] ERROR - VFSTransportListener cannot resolve fileObject org.apache.commons.vfs.FileSystemException: Badly formed URI . at org.apache.commons.vfs.provider.url.UrlFileProvider.findFile(UrlFileProvider.java:85) at org.apache.commons.vfs.impl.DefaultFileSystemManager.resolveFile(DefaultFileSystemManager .java:662) at org.apache.commons.vfs.impl.DefaultFileSystemManager.resolveFile(DefaultFileSystemManager .java:602) at org.apache.commons.vfs.impl.DefaultFileSystemManager.resolveFile(DefaultFileSystemManager .java:570) at org.apache.synapse.transport.vfs.VFSTransportListener.scanFileOrDirectory(VFSTransportLis tener.java:163) at org.apache.synapse.transport.vfs.VFSTransportListener.poll(VFSTransportListener.java:137) at org.apache.synapse.transport.vfs.VFSTransportListener.poll(VFSTransportListener.java:104) at org.apache.axis2.transport.base.AbstractPollingTransportListener$1$1.run(AbstractPollingT ransportListener.java:99) at org.apache.axis2.transport.base.threads.NativeWorkerPool$1.run(NativeWorkerPool.java:58) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:619) Caused by: java.net.MalformedURLException: unknown protocol: sftp at java.net.URL.(URL.java:574) at java.net.URL.(URL.java:464) at java.net.URL.(URL.java:413) at org.apache.commons.vfs.provider.url.UrlFileProvider.findFile(UrlFileProvider.java:66) ... 11 more [2009-11-11 09:57:49,291] ERROR - VFSTransportListener cannot resolve fileObject repeatedly: Badly f ormed URI . org.apache.commons.vfs.FileSystemException: Badly formed URI . at org.apache.commons.vfs.provider.url.UrlFileProvider.findFile(UrlFileProvider.java:85) at org.apache.commons.vfs.impl.DefaultFileSystemManager.resolveFile(DefaultFileSystemManager .java:662) at org.apache.commons.vfs.impl.DefaultFileSystemManager.resolveFile(DefaultFileSystemManager .java:602) at org.apache.commons.vfs.impl.DefaultFileSystemManager.resolveFile(DefaultFileSystemManager .java:570) at org.apache.synapse.transport.vfs.VFSTransportListener.scanFileOrDirectory(VFSTransportLis tener.java:163) at org.apache.synapse.transport.vfs.VFSTransportListener.poll(VFSTransportListener.java:137) at org.apache.synapse.transport.vfs.VFSTransportListener.poll(VFSTransportListener.java:104) at org.apache.axis2.transport.base.AbstractPollingTransportListener$1$1.run(AbstractPollingT ransportListener.java:99) at org.apache.axis2.transport.base.threads.NativeWorkerPool$1.run(NativeWorkerPool.java:58) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:619) Caused by: java.net.MalformedURLException: unknown protocol: sftp at java.net.URL.(URL.java:574) at java.net.URL.(URL.java:464) at java.net.URL.(URL.java:413) at org.apache.commons.vfs.provider.url.UrlFileProvider.findFile(UrlFileProvider.java:66) ... 11 more Can you please help me to resolve this?
bhaskar_annamalai.trimble.com's picture

attaching the synapse.xml

attaching the synapse.xml used Thanks
indika's picture

Could please check with [1].

Could please check with [1]. That user has used sftp successfully [1] http://wso2.org/forum/thread/5742
bhaskar_annamalai.trimble.com's picture

Hi, Thanks for your

Hi, Thanks for your inputs. I used a similar sftp URL. but it was not able to connect. i ran a sample program to connect the sftp url using vfs and got the same error. The error was resolved in my sample program by modifying the code to set the StrictHostKeyChecking to no and adding jsch jar in classpath. FileSystemOptions opts = new FileSystemOptions(); SftpFileSystemConfigBuilder.getInstance().setStrictHostKeyChecking(opts, "no"); FileObject file = fsManager.resolveFile(, opts); To apply the same change in wso2 esb, I changed the VFSTransportListener.java code to set the above option in case of an sftp url and tried to build wso2 esb. Is it possible to do the full product build using 'mvn clean install'? It does not seem to create a complete product zip file. Please suggest.
indika's picture

Nice to here that you have

Nice to here that you could get to working .... You can contribute this to Apache Synapse ... mvn clean install should work ..please checks on 'distribution' module Thanks Indika
bhaskar_annamalai.trimble.com's picture

Hi, Thanks Indika, i found

Hi, Thanks Indika, i found the wso2carbon zip under carbon-platform\carbon\distribution\target directory. But the vfs transport is not listed in axis2.xml by default. Is there any setting that needs to be done to include vfs transport? Thanks, Bhaskar
indika's picture

Nope , You have to manually

Nope , You have to manually edit it....
bhaskar_annamalai.trimble.com's picture

Hi, Thanks Indika. I

Hi, Thanks Indika. I modified the following files and built esb and tested a VFS proxy service. VFSTransportListener.java (set strict host key checking to 'no' if the url is an sftp url) product/modules/distribution/pom.xml (added jsch jar as dependency) product/modules/distribution/src/main/assembly/dist.xml (included jsch.jar under common outputDirectory) The VFS uploads the file to the sftp url. But the process errors out finally with the following error. VFS could not set the last modified timestamp of a sftp url file. Is there any specific reason for this issue? org.apache.commons.vfs.FileSystemException: Could not set the last modified timestamp of . at org.apache.commons.vfs.provider.DefaultFileContent.setLastModifiedTime(DefaultFileContent .java:196) at org.apache.commons.vfs.provider.AbstractFileObject.moveTo(AbstractFileObject.java:1001) at org.apache.synapse.transport.vfs.VFSTransportListener.moveOrDeleteAfterProcessing(VFSTran sportListener.java:331) at org.apache.synapse.transport.vfs.VFSTransportListener.scanFileOrDirectory(VFSTransportLis tener.java:262) at org.apache.synapse.transport.vfs.VFSTransportListener.poll(VFSTransportListener.java:154) at org.apache.synapse.transport.vfs.VFSTransportListener.poll(VFSTransportListener.java:117) at org.apache.axis2.transport.base.AbstractPollingTransportListener$1$1.run(AbstractPollingT ransportListener.java:99) at org.apache.axis2.transport.base.threads.NativeWorkerPool$1.run(NativeWorkerPool.java:58) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:619) Caused by: 3: Permission denied at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289) at com.jcraft.jsch.ChannelSftp._setStat(ChannelSftp.java:1871) at com.jcraft.jsch.ChannelSftp.setStat(ChannelSftp.java:1845) at org.apache.commons.vfs.provider.sftp.SftpFileObject.doSetLastModifiedTime(SftpFileObject. java:237) at org.apache.commons.vfs.provider.AbstractFileObject.doSetLastModTime(AbstractFileObject.ja va:262) at org.apache.commons.vfs.provider.DefaultFileContent.setLastModifiedTime(DefaultFileContent .java:189) ... 10 more I commented the code in vfs jar which sets the last updated time of the file after uploading it (for sftp alone). Now the above error did not occur. Will this create an issue? Or is there anything which i am missing here? Thanks, Bhaskar
rajika's picture

There is a bug in ESB which

There is a bug in ESB which stop using vfs transport for sftp support. This is fixed in trunk and the latest releases. Rajika
library project main code
Learn Cloud
Learn
Cloud

The WSO2 Application Server is a reliable application server that can host your enterprise web applications. The WSO2 Application Server as a Service is offered in StratosLive, the WSO2 Platform as a Service. This article explains how a simple web application can be developed and deployed from Carbon Studio to the WSO2 Application Server...

Latest Webinar
Different groups within an organization need to monitor different Key Performance Indicators (KPIs) - An operations team will be interested in the response times of business services and loads of each service,..
Thursday, February 9th 2012, 09.00 AM (PST)

Thursday, February 9th 2012, 10.00 AM (GMT)