Ziping Liu

At GaiaVerse, I lead the development of AI-driven platforms that empower organizations to navigate complex, global challenges by transforming data into actionable insights. My work focuses on creating technology that integrates environmental, social, and cultural data to support informed, sustainable decision-making. One of the key projects I’ve pioneered is GaiaGraph, an advanced graph database housing over 13 million records, designed to reveal patterns, relationships, and insights within fragmented data to guide responsible action.

In a recent significant technical breakthrough, I independently developed an AI-powered natural language query and visualization interface for GaiaGraph, without relying on other AI frameworks or tools. Within a few weeks, I delivered a feature that markedly enhances the capabilities of AI-driven data systems. This interface enables users to pose complex, conversational queries that are dynamically parsed and processed into data-backed answers, complete with intuitive visualizations.

Utilizing an advanced AI agent framework, I designed the system to manage complex questions by categorizing query complexity and breaking down advanced queries into sub-queries for efficient, concurrent processing. Each element of this interface—parallel query decomposition, contextual embeddings, vectorized retrieval, and keyword scoring—embodies a cutting-edge approach in AI for knowledge graphs.

The outcome is an interactive, decision-support tool that makes GaiaGraph both accessible and impactful for technical and non-technical users alike. This achievement underscores my expertise in advanced natural language processing, memory-efficient query handling, and real-time data visualization, positioning GaiaGraph as an exceptionally powerful resource within the industry.

Neo4j Integration and Retrieval-Augmented Generation (RAG)

GaiaGraph operates on Neo4j, a graph database that excels at representing and querying interconnected data but presents unique challenges due to its graph-centric architecture. To optimize Neo4j’s power, I developed a retrieval-augmented generation (RAG) system that intelligently retrieves, processes, and presents Neo4j data in response to user queries. The system’s embedding-based retrieval mechanism leverages Neo4j’s vast data through an AI pipeline that uses contextual embeddings, cosine similarity, and caching to create a responsive, real-time query experience. The natural language interface converts user queries into embeddings using OpenAI’s embedding model, transforming input into vectorized form for cosine similarity scoring. This allows the AI agent to identify and rank relevant data chunks from Neo4j with high accuracy, significantly enhancing the quality of responses.

To further refine retrieval, I developed a multi-layered approach involving keyword scoring and dynamic chunk refinement. For instance, Neo4j documentation is parsed into manageable chunks stored in a PostgreSQL database, each chunk pre-embedded as vectors to ensure rapid matching. This data is scored based on both cosine similarity and keyword relevance, prioritizing Neo4j-specific terms to enhance relevance. By integrating configurable weights for cosine similarity, domain-specific keywords, and Cypher terms, I ensured that the assistant’s responses are aligned with both the query’s intent and Neo4j’s domain.

Embedding Memory, Caching, and Zero-Shot Model Integration

One of GaiaGraph’s core innovations is the Persistent Knowledge Memory, a feature that combines embedding storage, query caching, and zero-shot models to support efficient, contextually aware responses. When processing queries, the system first checks for cached embeddings based on unique query hashes, accelerating response time and minimizing redundant processing. If no cached embedding is found, the assistant uses OpenAI’s text-embedding-3-large model to create a 512-dimensional vector representation of the query, which is then stored with metadata for future reference. This approach optimizes memory usage, reduces API costs, and enables the assistant to provide quick, accurate answers by recalling past context without reprocessing data.

To further enhance the system’s ability to handle diverse user queries, I implemented local zero-shot and task-specific models for data augmentation, categorizing column data within GaiaGraph entries. These models allow the system to dynamically augment data by generating new insights or classifications, particularly useful for categorizing violation data and metadata associated with different nodes in Neo4j. By integrating task-specific models locally, GaiaGraph maintains flexibility in responding to varied data requests while enriching the scope of responses.

Advanced Query and Chunk Refinement with Embeddings

GaiaGraph’s chunk refinement process represents an advanced application of dynamic similarity-based chunking and local coherence refinement. Neo4j documentation is divided into chunks based on token limits, initially processed for broad thematic coherence and then refined for local semantic alignment. The refinement process utilizes sentence-transformers models to evaluate intra-chunk and inter-chunk similarity. Sentences are shifted between chunks based on cosine similarity, creating coherent thematic sections that enhance embedding accuracy. Each refined chunk is embedded and stored, enabling precise retrieval based on user queries.

The embedding generation for each chunk includes multiple passes, each adjusting sentence placement for thematic continuity. By combining dynamic similarity refinement and local similarity checks, the system achieves maximum intra-chunk coherence, ensuring that retrieved chunks provide the most relevant context for generating Cypher responses. Each chunk embedding is cached and indexed in PostgreSQL, allowing for rapid retrieval, vectorized scoring, and prioritized response generation.

End-to-End AI Agent Framework for Advanced Data Interpretation

Through the seamless integration of query decomposition, memory-based retrieval, and RAG capabilities, GaiaGraph’s AI agent framework has transformed traditional data querying into an interactive experience that interprets, synthesizes, and visualizes complex information in real-time. The AI agent uses a structured, multi-step process that includes:

• Categorizing query complexity to determine whether direct processing or sub-query decomposition is needed.
• Embedding-based retrieval to match user intent with the most relevant data within Neo4j.
• Parallel sub-query processing to handle decomposed queries efficiently, storing results from concurrent processing for final synthesis.
• Combined scoring mechanisms (cosine similarity, keyword weighting) to prioritize chunks that align both semantically and contextually with the query.

This approach, combined with intuitive visualization, empowers GaiaGraph to deliver cohesive, meaningful insights by synthesizing responses across multiple data points.

Ethical and Sustainable AI Development

GaiaVerse is built on the principle that AI must be consciously guided to promote ethical and sustainable outcomes. My work aligns deeply with this mission, as I am committed to creating AI systems that not only advance technology but also prioritize responsible development. Each layer of GaiaGraph reflects a balance between innovation and ethical considerations, ensuring that AI is used to empower decision-making in ways that support long-term environmental stewardship. By transforming complex data into accessible insights, GaiaGraph embodies GaiaVerse’s vision for an AI system grounded in sustainability, resilience, and a commitment to the greater good.

Through this work, I contribute to GaiaVerse’s mission of helping organizations adapt responsibly to a changing world and become effective, conscientious stewards of the planet. The technical depth, scalability, and intentionality behind GaiaGraph set a new standard for how AI systems can interact with data, embodying the future of ethical, responsible AI in data science and decision support. If you have any questions or would like to explore these concepts further, feel free to ask my chatbot, Synthethics, found at the bottom of the page as an expandable chat window, where it’s ready to assist with in-depth insights and details on GaiaVerse projects.