[Registry-dev] Integrating Apache Lucene with WSO2 Registry

[Deepal Jayasinghe]

Seems good to me , have you integrate your code with the trunk or branch ?

Thank you!
Deepal

kalani wrote:
> hi,
>
> I have completed the sample application which includes creating the
> index in the database, loading index into the memory, searching and
> periodically flushing index updates to the database. 
>
> The plan is to write a handler to manage the index and to introduce a
> new method to the Registry interface for content searching in order to
> integrate this with the Registry.
>
> Thanks,
> Kalani  
>
> On Fri, 2008-06-20 at 04:27 +0530, kalani wrote:
>   
>> hi Paul,
>>
>> I considered about the JVM clustering option using Terracotta framework
>> which is mentioned in the 1st link. However I think it is not necessary
>> to go that far if we can create the index in the database (I talked
>> about this issue with Azeez too). 
>>
>> The Compass project which is under Apache license provides an
>> implementation of JdbcDirectory. We can use that implementation to get
>> the thing done. To overcome the performance issues we can load the index
>> to a RamDirectory and periodically flush updates to the database. At the
>> moment I am working on a sample application and I think this way is easy
>> and hope it will work fine. 
>>
>> Thanks,
>>
>> Kalani
>>
>> On Mon, 2008-06-09 at 15:42 +0100, Paul Fremantle wrote:
>>     
>>> You might want to read this blog:
>>>
>>> http://orionl.blogspot.com/2006/11/clustering-lucene.html
>>>
>>> and this:
>>>
>>> http://bugs.sakaiproject.org/confluence/display/SEARCH/IndexClusterOperation
>>>
>>> Regards
>>>
>>> Paul
>>>
>>> kalani wrote:
>>>       
>>>> Hi Paul,
>>>>
>>>> I got your point and did a search on working with Lucene in a clustered
>>>> environment.
>>>>
>>>> A possible option is to store the index inside the shared database.
>>>> Lucene, however, does not directly support this feature [1]. I will try
>>>> to find an implementation of the suggested JdbcDirectory class as well.
>>>> I have already sent a mail regarding this to Lucene user list.
>>>>
>>>> Thanks,
>>>>
>>>> Kalani
>>>>
>>>> [1]http://wiki.apache.org/lucene-java/LuceneFAQ#head-e55d8e6971f9f01daaf3e14ce1d2f34485adba6e
>>>>
>>>> On Mon, 2008-06-09 at 09:12 +0100, Paul Fremantle wrote:
>>>>         
>>>>> Kalani
>>>>>
>>>>> The issue is that if you create the index in a local file system, you 
>>>>> make the overall system harder to cluster.
>>>>>
>>>>> This is because - in a typical cluster scenario - a shared database is a 
>>>>> standard capability and will already be there. A shared file system adds 
>>>>> extra complexity.
>>>>>
>>>>> Please can you find out if there is any guidance on running Lucene in a 
>>>>> cluster. Thanks!
>>>>>
>>>>> Paul
>>>>>
>>>>> kalani wrote:
>>>>>           
>>>>>> Hi all,
>>>>>>
>>>>>> I am working on implementing the content search functionality of the 
>>>>>> Registry using Apache Lucene.
>>>>>>
>>>>>> Apache Lucene can easily be used to search content by indexing the files
>>>>>> inside the file system. Additionally,  a database can also be indexed by
>>>>>> adding each and every record to an index. The index should not be
>>>>>> recreated at each time a new resource is added to the database. The new
>>>>>> resource should be indexed and appended to the existing index at the
>>>>>> time it is added.
>>>>>>
>>>>>> The index is created in the local file system. The Lucene project
>>>>>> mentions that there is a performance issue in creating the index in a
>>>>>> database. However, they have a mechanism called Directory to support it.
>>>>>> We can look into that later if needed.
>>>>>>
>>>>>> Thanks
>>>>>>
>>>>>> Kalani
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Registry-dev mailing list
>>>>>> Registry-dev at wso2.org
>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/registry-dev
>>>>>>
>>>>>>             
>>>> _______________________________________________
>>>> Registry-dev mailing list
>>>> Registry-dev at wso2.org
>>>> http://wso2.org/cgi-bin/mailman/listinfo/registry-dev
>>>>
>>>>         
>> _______________________________________________
>> Registry-dev mailing list
>> Registry-dev at wso2.org
>> http://wso2.org/cgi-bin/mailman/listinfo/registry-dev
>>     
>
>
> _______________________________________________
> Registry-dev mailing list
> Registry-dev at wso2.org
> http://wso2.org/cgi-bin/mailman/listinfo/registry-dev
>
>
>   

-- 
Thanks,
Deepal
................................................................
http://blogs.deepal.org/