DiploCloud: Efficient and Scalable Management of RDF Data in the Cloud

ABSTRACT:

Despite recent advances in distributed RDF data management, processing large-amounts of RDF data in the cloud is still very challenging. In spite of its seemingly simple data model, RDF actually encodes rich and complex graphs mixing both instance and schema-level data. Sharding such data using classical techniques or partitioning the graph using traditional min-cut algorithms leads to very inefficient distributed operations and to a high number of joins. In this paper, we describe DiploCloud, an efficient and scalable distributed RDF data management system for the cloud. Contrary to previous approaches, DiploCloud runs a physiological analysis of both instance and schema information prior to partitioning the data. In this paper, we describe the architecture of DiploCloud, its main data structures, as well as the new algorithms we use to partition and distribute data. We also present an extensive evaluation of DiploCloud showing that our system is often two orders of magnitude faster than state-of-the-art systems on standard workloads.

EXISTING SYSTEM:

• While much more recent than relational data management, RDF data management has borrowed many relational techniques; Many RDF systems rely on hash-partitioning and on distributed selections, projections, and joins.
• Grid-Vine system was one of the first systems to do so in the context of large-scale decentralized RDF management.
• Approaches for storing RDF data can be broadly categorized in three subcategories: triple-table approaches, property-table approaches, and graph-based approaches.
• Hexastore suggests to index RDF data using six possible indices, one for each permutation of the set of columns in the triple table. RDF-3X and YARS follow a similar approach.
• BitMat maintains a three-dimensional bit-cube where each cell represents a unique triple and the cell value denotes presence or absence of the triple. Various techniques propose to speed-up RDF query processing by considering structures clustering RDF data based on their properties.

DISADVANTAGES OF EXISTING SYSTEM:

• Existing system generates much inter-process traffic, given that related triples (e.g.,that must be selected and then joined) end up being scattered on all machines.
• RDF actually encodes rich and complex graphs mixing both instance and schema-level data. Sharding such data using classical techniques or partitioning the graph using traditional min-cut algorithms leads to very inefficient distributed operations and to a high number of joins.
• Existing system are not efficient and not scalable system for managing RDF data in the cloud.
• In existing system lot of complex while query execution.
• Existing system are slower while handling the standard workloads.
• 
  

PROPOSED SYSTEM:

• In this article, we propose DiploCloud, an efficient, distributed and scalable RDF data processing system for distributed and cloud environments. Contrary to many distributed systems, DiploCloud uses a resolutely non-relational storage format, where semantically related data patterns are mined both from the instance-level and the schema-level data and get co-located to minimize internode operations. The main contributions of this article are:
• A new hybrid storage model that efficiently and effectively partitions an RDF graph and physically co-locates related instance data;
• A new system architecture for handling fine-grained RDF partitions in large-scale
• Novel data placement techniques to co-locate semantically related pieces of data
• New data loading and query execution strategies taking advantage of our system’s data partitions and indices
• An extensive experimental evaluation showing that our system is often two orders of magnitude faster than state-of-the-art systems on standard workloads

ADVANTAGES OF PROPOSED SYSTEM:

• DiploCloud is an efficient and scalable system for managing RDF data in the cloud.
• DiploCloud is particularly suited to clusters of commodity machines and cloud environments where network latencies can be high, since it systematically tries to avoid all complex and distributed operations for query execution.

SYSTEM ARCHITECTURE:

SYSTEM REQUIREMENTS:

HARDWARE REQUIREMENTS:

• System                           :         Pentium Dual Core.

• Hard Disk                      :         120 GB.

• Monitor                         :         15’’ LED

• Input Devices                 :         Keyboard, Mouse

• Ram                               :         1GB 

SOFTWARE REQUIREMENTS:

• Operating system    :         Windows 7.
• Coding Language           :         JAVA/J2EE
• Tool                               :         Netbeans 7.2.1
• Database                        :         MYSQL

REFERENCE:

Marcin Wylot and Philippe Cudr_e-Mauroux, “DiploCloud: Efficient and Scalable Management of RDF Data in the Cloud”, IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, VOL. 28, NO. 3, MARCH 2016.