Too many open files after a while

aldo.strac.gmail.com's picture

Hi, I've deployed WSO2 Mashup Server and into it I've deployed a periodical mashup wich invokes other task and use their results; after a while (let's say about 12-24 hours) my services stop working and became marked as faulty. Looking in the console I see that there are lots of IOExceptions caused by "Too many open files". I've looked at open files in my system (debian lenny amd64) and I've got the following results (filtered only the one containing "wso2"):

pastebin.com/m76367d9

There seems to be a lot of processes opened in a temporary folder belonging to Axis2, can someone tell me how to work around this issue? I've got it with Mashup server version 2.0.1; I'm trying now with version 2.0.2, I'll post my results.

tyrell's picture

As a quick fix you can

As a quick fix you can increase the number of files allowed to be opened in your linux system. The following link should help you. http://www.cs.wisc.edu/condor/condorg/linux_scalability.html   Tyrell  
aldo.strac.gmail.com's picture

At this moment I have 74854

At this moment I have 74854 as maximum open files; the system is a stripped-down debian amd64 server running only the OS, the mashup server and a simple web application deployed in tomcat. I think that raising that limit will not fix the problem, simply it will show after more time, but I'll give it a try and post my results. In my opinion there is a bug that imply not closing temporary files created by axis2.
tyrell's picture

Yes this seems like a bug.

Yes this seems like a bug. We are working on figuring out where it originates exactly. Tyrell
aldo.strac.gmail.com's picture

In hope it will be useful

In hope it will be useful. I'm posting in attachement another file containing the result of $ lsof | grep wso2 with the new open file limit and version 2.0.2 of mashup server. I hope to get a fix for this since it will be critical to my work.
library project main code
Learn Cloud
Learn
Cloud

The WSO2 Application Server is a reliable application server that can host your enterprise web applications. The WSO2 Application Server as a Service is offered in StratosLive, the WSO2 Platform as a Service. This article explains how a simple web application can be developed and deployed from Carbon Studio to the WSO2 Application Server...

Latest Webinar
Different groups within an organization need to monitor different Key Performance Indicators (KPIs) - An operations team will be interested in the response times of business services and loads of each service,..
Thursday, February 9th 2012, 09.00 AM (PST)

Thursday, February 9th 2012, 10.00 AM (GMT)