|Zet Universe Technical Requirements Document|
[This is preliminary documentation and is subject to change.]
This topic contains the following sections:
Zet Universe platform (cloud + client) enables Data Integration, Organization, Search, and Analysis of the end user and enterprise content. The following essential functions are required:
[both] Store user-defined metadata (projects) and data (from external data sources)
[both] Aggregate data from external data sources into user-defined projects
[both] Analyze (derive and extract) meaningful kinds and properties from data
[both] Index kinds and properties
[both] Annotate kinds with (semantic) markup
[client] Search kinds using terms, phrases, time intervals, locations, and exemplars
[both] Enable collaborative work on user-defined projects
[cloud] Map user-defined knowledge graphs together
[cloud] Build and grow unified enterprise knowledge graph
[both] Manage system state, configuration, performance, and other administrative tasks
[both] Act on kinds in semantically appropriate ways
The Zet Universe is built from a small foundation of concepts: Agents, Models, Entities and Kinds, Properties, Relationships, Apps, and Processors.
Agents represent real people, as well as artificial intelligence systems that operate within the Zet Universe platform. Artificial intelligence systems are generally represented as the installed apps/accounts.
Models are Agent- or App-generated containers designed to be a place for data integration, organization, and analysis. For the end users, Models can be seen as the workspaces for their projects, while Apps use Models to obtain and synchronize external data, to make it available for the end users. Models contain collections of instances of Kinds.
Knowledge within the Zet Universe is represented as entities on storage level and kinds on ontology level.
Entity is the "atom" of information within Zet Universe. It incapsulates information about any kind of the everyday things like people, documents, e-mail, and so on. A special property, Kind, is used to characterize the semantic meaning of the particular Entity. Entities are extracted and/or derived from the provided data sources or emergent from collections of entities and properties.
While Entity is a practical implementation of the Zet Universe foundational data type, Kind could be seen as both a classification term used to describe a particular Entity, and as an element of the ontology which is a mechanism used to build a categorization of the everyday things, as well as the specific domains (think Customers in CRM). Kinds are flexible representations of both every day and enterprise-specific things: people, organizations, places, events, media, etc.
Derived, extracted, or user-supplied data about kinds and/or their underlying data sources are provided in two forms: common properties (available for all Kinds), and dynamic properties.
Relationships represent connections between instances of Kinds, and, like Properties, they are derived, extracted, or user-supplied. Relationships are stored with the instances of Kinds as references to other instances.
Content is provided to the system via external data providers (apps) and is represented by a Kind. Apps connect Zet Universe to the external data sources (local and network file systems, web, microservices, etc.), enable initial data transformation and load, as well as allow (optional) source change tracking, and (optional) metadata synchronization. App may register own Processors (see below), Kinds’ and Relationships’ definitions. Support is provided (minimally) for file-based documents (local and Dropbox-based), images, audio, video, people (Dropbox users), web (HTML).
Additional kinds, apps and processors can be added to the system.
Content within Kinds is processed via the means of the so-called Semantic Pipeline (see below), which provides a topic subscription service, and both the platform and plugins can subscribe to this service, specify a topic (or topics) they are interested in, process incoming instances of Kinds, and (optionally) publish updated instances of Kinds back to the Semantic Pipeline. There are two kinds of subscribers – lightweight (that can process data in an almost synchronous way) and heavy (that usually spend significant time to analyze and process data). Both kinds of subscribers are usually defined in the form of Processors, that are packaged either as part of Apps, or independently.
The Zet Universe platform is deployed as a combination of cloud and client systems working in a tandem. Cloud and client systems work together via a combination of REST application interfaces (for non-critical data synchronization) and Web Sockets (for near real-time collaboration).
Current (Windows-based) and future (Web, mobile, AR/VR) client applications are the end user interface to the Zet Universe Platform. Two kinds of clients, lightweight and full, will be available as a part of the platform:
Full clients include both the user interface and everything necessary for offline data integration, indexing, and analysis
Lightweight clients provide only the user interface and can be (optionally) enhanced by the additional plugins.
Cloud component of the Zet Universe platform is a (scalable) unit deployable in both private and public clouds. Its primary goals are to provide:
Metadata storage for storing knowledge represented in the form of Kinds.
BLOB storage for storing supplemental (thumbnails, etc.) to Kinds and (optionally) external content (files, emails, etc.).
Near real-time subscription service to enable seamless collaboration options for Agents working with the shared projects Models.
Simple authorization and authentication subsystem (OAuth 2.0-based).
Semantic pipeline and Processors infrastructure for data analysis (deriving and extracting Kinds, Properties, and Relationships).
Apps infrastructure for external data extraction, load, and transformation (ETL).
Ontology management infrastructure for graphs generation, editing, growth, and mapping based on the individual Agents' contribution via their own projects (Models) in the "Friend-to-Friend" way.
Zet Universe is an innovation platform (cloud + client) that enables Data Integration, Organization, Search, and Analysis of enterprise content.