escience graph

Enabling Data Transport between Web Services

Despite numerous benefits, many Web Services (WS) face problems with respect to data transport, either because SOAP doesn’t offer a scalable way of transporting large data-sets or because orchestration workflows (WF) don’t move data around efficiently. In this paper we address both problems with the development of the ProxyWS. This is a WS utilizing protocols offered by the Virtual Resource System (VRS), to enable other WS to transfer and access large datasets without modifying WS nor the underlying environment.

There is currently an abundance of deployed (legacy) WS using SOAP, which fail to produce access and return large datasets. Moreover, orchestration WF causes WS to pass messages containing data back through the WF engine. To address these problems we introduce the ProxyWS: a WS that is able to access data from remote resources (GridFTP, LFC, etc.), thanks to the VRS, and also transport larger data produced by WS, both legacy and new. For the ProxyWS to be able to provide larger data transfers to legacy WS, it has to be deployed on the same Axis-based container, just like a normal WS. This enables clients to make proxy calls to the ProxyWS instead of a legacy WS. As a consequence the ProxyWS returns a SOAP message containing a URI referring to the data location. For new implementations the ProxyWS is used as an API that can create data streams from remote data resources and other WS using the ProxyWS. This approach proved to be the most scalable since WS can process data as they are generated from producing WS. Thus with the introduction of the ProxyWS we are able to provide a separate channel for data transfers, that allows for more scalable SOA-based applications.

Many different approaches have been introduced in an attempt to address the problems mentioned earlier. Examples of these include Styx Grid Services, Data Proxy Web services for Taverna and Flex-SwA. Some noteworthy features of these approaches are: Direct streaming between WS, Usage of alternative protocols for data transports, and larger data delivery to legacy WS. However, each of these examples only addresses one part of the problem and, furthermore, do not include any means of allowing access to remote data resources. Leveraging these existing proposals and combining them with the VRS we implemented a ProxyWS. To validate it, we have tested its performance using 2 data-intensive WF. The first is a distributed indexing application that uses a set of WS to speedup the indexing of a large set of documents, while the second relies on the creation of that index for retrieving and recognizing protein names contained in results coming from a query. With the use of the ProxyWS we are able to retrieve data from remote locations (8.4 GB of documents for indexing), as well as to obtain more results relative to a query (8300 documents using the ProxyWS versus 1100 using SOAP).

We have presented the ProxyWS, which may be used to support large data transfers for legacy and new WS. We have verified its performance to deliver large datasets on two real-life tasks: Indexing using WS in a distributed environment and annotating documents from an index. From our experiments we have found that ProxyWS is able to facilitate data transports where normal SOAP messages would have failed. We have also demonstrated that with the use of the ProxyWS legacy WS can scale further, by avoiding data delivery via SOAP and by delivering data directly from the producing to the consuming WS.

  • [PDF] S. Koulouzis, E. Meij, and A. Belloum, “Enabling large data transfers between web services,” in 5th egee user forum, 2010.
    Author = {Koulouzis, S. and Meij, E. and Belloum, A.},
    Booktitle = {5th EGEE User Forum},
    Date-Added = {2011-10-20 10:00:08 +0200},
    Date-Modified = {2011-10-20 10:00:08 +0200},
    Title = {Enabling Large Data Transfers Between Web Services},
    Year = {2010}}
Annals of Information Systems

Semantic disclosure in an e-Science environment

The Virtual Laboratory for e-Science (VL-e) project serves as a backdrop for the ideas described in this chapter. VL-e is a project with academic and industrial partners where e-science has been applied to several domains of scientific research. Adaptive Information Disclosure (AID), a subprogram within VL-e, is a multi-disciplinary group that concentrates expertise in information extraction, machine learning, and Semantic Web – a powerful combination of technologies that can be used to extract and store knowledge in a Semantic Web framework. In this chapter, the authors explain what “semantic disclosure” means and how it is essential to knowledge sharing in e-Science. The authors describe several Semantic Web applications and how they were built using components of the AIDA Toolkit (AID Application Toolkit). The lessons learned and the future of e-Science are also discussed.

  • [PDF] M. S. Marshall, M. Roos, E. Meij, S. Katrenko, W. R. van Hage, and P. W. Adriaans, “Semantic disclosure in an e-science environment,” in Semantic e-science (springer annals of information systems aois), 2009.
    Author = {Marshall, M.S. and Roos, M. and Meij, E. and Katrenko, S. and van Hage, W.R. and Adriaans, P.W.},
    Booktitle = {Semantic e-Science (Springer Annals of Information Systems AoIS)},
    Date-Added = {2011-10-16 15:03:17 +0200},
    Date-Modified = {2012-10-28 17:21:26 +0000},
    Publisher = {Springer},
    Series = {Annals of Information Systems},
    Title = {Semantic disclosure in an e-Science environment},
    Volume = {11},
    Year = {2009}}
AGRO informatica

De Aida toolbox: Een gecombineerde aanpak voor het beheren van kennis

In een computationele netwerk omgeving zoals het grid is een overvloed aan zeer uiteenlopende soorten bronnen aanwezig. Denk bijvoorbeeld aan tijdschrift artikelen, beelden, massa spectrometrie data, R scripts voor statistiek, web services, workflows of spreadsheets. Deze overvloed kan een grote belemmering vormen. Hoe moet een gebruiker de juiste bronnen vinden voor een voorliggend probleem? Vele factoren maken het matchen van de benodigdheden en gebruikerswensen aan wat de bronnen kunnen leveren en de regels ten aanzien van hun gebruik een complex probleem. Het probleem doet zich voor op verschillende niveaus. Eindgebruikers willen het benodigde vinden in hun eigen domein. Applicatie en middelware ontwikkelaars moeten services en data kunnen vinden, bij voorkeur geautomatiseerd zodat veranderingen in aanwezigheid en toegankelijkheid kunnen worden opgevangen. Dit probleem beperkt zich niet tot grids; ook het Web en allerlei dataopslag toepassingen hebben er mee te maken. Ook voor ‘enhanced science’ (e-science) is het beheren van heterogene bronnen een belangrijke uitdaging.

  • [PDF] M. S. Marshall, M. Roos, E. Meij, S. Katrenko, W. R. van Hage, and P. W. Adriaans, “De AIDA toolbox: een gecombineerde aanpak voor het beheren van kennis,” Agro informatica, vol. 21, iss. 4, pp. 5-7, 2009.
    Author = {Marshall, M.S. and Roos, M. and Meij, Edgar and Katrenko, S. and van Hage, W.R. and Adriaans, P.W.},
    Date-Added = {2011-10-16 15:55:36 +0200},
    Date-Modified = {2012-10-28 23:04:41 +0000},
    Edition = {1},
    Journal = {Agro Informatica},
    Number = {4},
    Pages = {5--7},
    Title = {De {AIDA} toolbox: Een gecombineerde aanpak voor het beheren van kennis},
    Volume = {21},
    Year = {2009}}
workflow process

Biological applications of Aida knowledge management components

Given the important role of knowledge in biology, knowledge in a machine readable form can be an important asset for bioinformatics. We present two applications of AIDA (Adaptive Information Disclosure Application), a collection of knowledge management components. One is a workflow that extends a semantic model with putative relations between proteins and diseases extracted from literature by machine learning techniques. The other extends vBrowser, a virtual resource browser tool, with the ability to find relevant biological resources (e.g. data, workflows, documents) via semantic relationships.

Central to our semantic web approach is the separation of a ‘virtual knowledge space’ from its applications. In other words, knowledge is disclosed and accessed in a knowledge space rather than being coded into the application. The workflow adds knowledge to this space with knowledge extraction, while vBrowser accesses the knowledge resources for use during search. We use RDF and OWL to represent knowledge and Sesame to store RDF and OWL representations of knowledge.

The workflow contains the following steps: (i) add the ontology that you want to extend to Sesame (e.g. a model that contains the protein EZH2), (ii) extract the entities of interest from the ontology (e.g. EZH2), (iii) retrieve abstracts from Medline for these entities, (iv) extract proteins and protein-protein relationships from the abstracts, (v) add a ranking score to the discoveries, (vi) query OMIM with the extracted proteins and retrieve the disease labels (service from the National Institute of Genetics in Japan), (vii) add the discoveries and their interrelationships to the repository, (viii) export the enriched ontology to the knowledge space where for instance vBrowser can be used to explore the results. Future work includes metrics to more effectively retrieve biologically interesting suggestions from semantic data.

We show how the vBrowser can be used to browse both data resources and knowledge resources from the same basic interface. We show how vBrowser uses an AIDA thesaurus service to improve finding resources such as Medline documents and workflows on We found thesauri terms effective for search and advocate SKOS for its intuitive ‘broader/narrower-than’ relationships. We further show that the protein-disease relationships resulting from our knowledge capture workflow as well as the documents that contained these relationships can be accessed as knowledge resources from the vBrowser. We think OWL can adequately represent the knowledge in many biological cartoon models and have used it to represent the workflow provenance in our knowledge capture workflow.

  • M. Roos, S. M. Marshall, P. T. de Boer, K. van den Berg, S. Katrenko, E. Meij, W. R. van Hage, and P. W. Adriaans, “Biological applications of AIDA knowledge management components,” in Ismb ’08, 2008.
    Author = {Marco Roos and M. Scott Marshall and Piter T. de Boer and Kasper van den Berg and Sophia Katrenko and Edgar Meij and Willem R. van Hage and Pieter W. Adriaans},
    Booktitle = {ISMB '08},
    Date-Added = {2011-10-16 10:45:35 +0200},
    Date-Modified = {2012-10-28 23:04:46 +0000},
    Title = {Biological applications of {AIDA} knowledge management components},
    Year = {2008}}
escience graph

Enabling Data Transport between Web Services through alternative protocols and Streaming

As web services gain acceptance in the e-Science community, some of their shortcomings have begun to appear. A significant challenge is to find reliable and efficient methods to transfer large data between web services. This paper describes the problem of scalable data transport between web services, and proposes a solution: the development of a modular Server/Client library that uses SOAP as a control channel while the actual data transport is accomplished by various protocol implementation, as well as a simple API that developers can use for data-intensive applications. Apart from file transport, the proposed approach offers the facility of direct data streaming between web services, an approach that could benefit workflow execution time by creating a data pipeline between web services. Finally, the performance and usability of this library is evaluated, under the indexing application that the Adaptive Information Disclosure Application (AIDA) Toolkit offers as a Web Service.

  • [PDF] S. Koulouzis, E. Meij, M. S. Marshall, and A. Belloum, “Enabling data transport between web services through alternative protocols and streaming,” in 4th ieee international conference on e-science, 2008.
    Author = {Koulouzis, S. and Meij, E. and Marshall, M.S. and Belloum, A.},
    Booktitle = {4th IEEE International Conference on e-Science},
    Date-Added = {2011-10-16 10:35:31 +0200},
    Date-Modified = {2011-10-16 10:35:31 +0200},
    Title = {Enabling Data Transport between Web Services through alternative protocols and Streaming},
    Year = {2008}}

My first BioAID: heuristic support for hypothesis construction from literature


Constructing a new hypothesis is often the first step for a new cycle of experiments. A typical approach to harvesting biological literature is to scan the results of a PubMed query and read what we think is most relevant. In this scenario, we are limited by the selection of papers and, for future applications, we are limited by our capacity to recall the knowledge we have gained. As part of the development of a ‘virtual laboratory for bioinformatics,’ we seek alternative ways to support the construction of hypotheses from biological literature.


Our objective is to provide automated support for hypothesis formation from literature based on an initial seed of knowledge.


Our approach consists of the following steps: first we create a ‘proto-ontology’ from the knowledge that we want to extend, for instance, a table in a review that lists diseases associated with a particular enzyme. We then identify the collection of documents that we want to search (typically Medline). Subsequently, we use concepts from our proto-ontology as input to retrieve relevant documents from a collection and to inform us of concepts such as protein names or relationships that are putatively associated with the proto- ontology. These results are used to enrich the proto-ontology with additional concepts and relations. The ontology can be iteratively enriched by using the results from one run as input for the next.


Our implementation is based on a collection of web services, allowing us to construct custom workflows for specific tasks. Together, these web services form a toolbox called AIDA (Adaptive Information Disclosure Application), for annotating documents, searching documents, discovering knowledge from documents, and storing ontological data. AIDA uses open source software such as Lucene for document retrieval, and Sesame for handling ontologies. For the purposes of this implementation, we have also used Taverna to construct our workflows and Protégé.


We have created workflows from services in the AIDA toolbox, and applied them to extend a proto- ontology with knowledge extracted from literature. Technically, the most challenging workflow uses our own proto-ontology as input for machine learning services, after which biological concepts are discovered that are related to terms from our own ontology. As a proof of concept, we have (re)discovered diseases that are known to be related to EZH2, an enzyme associated with gene regulation via chromatin remodelling. A second workflow which discovers genomics concepts is used to identify proteins that might present a previously unreported link between two biological concepts, e.g. histones and transcription factors. The proto-ontology and enriched ontology are written in the Web Ontology Language OWL, and stored in Sesame via another service from the toolbox.


Services and workflows are available from Ontologies are available from


Workflows constructed from the AIDA toolbox can be used as an aid in constructing hypotheses from literature. We show that we can automatically extend a proto-ontology with new hypothetical concepts and relationships that bridge across the boundaries of single papers or biological subdomains. Our approach can be customized to particular domains and vocabularies through the choice of ontology and literature corpora.

  • [PDF] M. Roos, S. Katrenko, W. R. van Hage, E. Meij, M. S. Marshall, and P. W. Adriaans, “My first bioaid: heuristic support for hypothesis construction,” in Ismb-eccb’07, 2007.
    Author = {Roos, M. and Katrenko, S. and van Hage, W.R. and Meij, E. and Marshall, M.S. and Adriaans, P.W.},
    Booktitle = {ISMB-ECCB'07},
    Date-Added = {2011-10-13 08:56:20 +0200},
    Date-Modified = {2011-10-13 08:56:20 +0200},
    Title = {My first BioAID: heuristic support for hypothesis construction},
    Year = {2007}}