[Registry-dev] Pagination of child resources lists and search result lists

[Chathura C. Ekanayake]

There are few problems in implementing the pagination in the registry. 
Below is the pagination API for collections.

public String[] getChildren(int start, int pageLen);

Users should be able to get the first 10 child resource paths of the 
collection c1 by calling,

c1.getChildren(1, 10);

Similarly it should give the 11th to 20th child resource paths for below 
call,

c1.getChildren(11, 10);

Now the problem is that it should *only* give the authorized paths for 
the current user and it should give *10* results as user has requested 
10 results.
Collection can get the the 1 to 10 child paths from the database. But 
then it has to check permissions of those paths and remove unauthorized 
paths. After removing the unauthorized paths resulting paths count can 
be below 10 (there can be no results left as well). We can solve this by 
fetching the missing number of paths (or with some buffer) again and 
checking the permissions. And doing this till we get 10 paths.

But the harder part comes next. When the user requested the paths 11th 
to 20th, the jdbc layer can't start with the 11th result. 11th path may 
have been included in the 1-10 set requested earlier. So far the 
solution I came up with for this is to keep track of mappings between 
requested result number -> jdbc level result number. Then, whenever a 
result number is requested, we can determine the jdbc level result 
number by the jdbc level result number of largest requested result 
number below the currently requested number. Below is an example:

Let's say user has requested,

c1.getChildren(1, 10);

Let's assume that the user doesn't have permissions for results 2, 5 and 
7. Then we have to fetch 3 additional results. So the mapping is:

request -> jdbc
1 -> 1
10 -> 13

Then user calls c1.getChildren(11, 10).

Now we know that the 11th result should actually begin with the jdbc 
level result 14. So internally we can ask for results 14 to 24.

Now if the user has requested result 15 to 25, we still have to start 
with jdbc level result 14 and skip 5 authorized results. But usually 
this does not occur as pages are browsed in sequential order.

All above complexity comes because it is very difficult (if not 
impossible) to perform authorizing algorithm in SQL. If we could perform 
authoring in same SQL, which is used to query resources, all the 
pagination operations can be performed in the jdbc level.

But for the current situation, I could only think of the above solution 
so far. This get more tricky, when it comes to pagination of query 
results and activity results. We can solve that by storing above maps 
against the hash of query path, paramaters, etc.

Thoughts...

Thanks,
Chathura