# User Guide ### What is GLOSI? GLOSI is an offering from Lineaje that serves as an intelligence data platform for open source components and vulnerabilities. GLOSI enables customers to lookup open source intelligence in their own environment. GLOSI supports a variety of environments – from cloud to on-prem. GLOSI releases new intelligence data on a regular basis to keep up the changes. ### How is GLOSI shipped? GLOSI is shipped as a container image (tarball) and it can be deployed using docker compose. ### What does GLOSI contain? GLOSI image bundles the open source data and open source vulnerability data along with a set of services and a UI. It includes the following elements: · Front end UI React Next App to access and query the open source components and vulnerabilities · Data-service FastAPI as an API layer to access the data · Orchestration service FastAPI app to manage the DB · Elastic Search DB `7.17.0` that holds the open source data ### How can I get access to GLOSI? GLOSI image is hosted in Lineaje registry. Please get in touch with Lineaje support () to get access to the GLOSI image. ### GLOSI Setup #### System Configuration All containers can be setup in one single VM (Ubuntu, Alpine) or in distributed fashion. If all applications are being setup in one single vm below configuration is recommended. | **Deployment Type** | **services** | **Ram** | **vCPU** | **DiskSize** | | ------------------- | ------------ | ------- | -------- | ------------ | | SingleNode | data-service | 32GB | 8 | 300GB | #### #### **Pre-requisite** 1\. You should have the tarball containing the installation guide, API documentation along with Resource folder containing the docker-compose.yaml file which contains GLOSI image hosted within Lineaje ECR. The tarball can also be found here. 2\. You should have access to Lineaje ECR hosting the GLOSI docker image, in case you don’t have access, then please reach out to Lineaje Support on for requesting ECR access along with your AWS Account Id. Once ECR access is granted on your AWS Account, you are good to move forward with next steps. 3\. Docker installed on the host vm 4\. Install AWS CLI on vm 5\. For automated installation, services being installed using the GLOSI vm should have access to [gold.lineaje.com](http://gold.lineaje.com) domain. Whitelisting #### **Deployment** 1\. Add a config in \~/.aws/config with below info ``` [profile LineajeGlosiRole] ``` ``` role_arn = arn:aws:iam:::role/lineajeGlosiPullRole ``` ``` credential_source = Ec2InstanceMetadata ``` 2\. Login to aws ecr using below command. glosi repository id can be found in docker-compose.yaml file present under Resource folder of tarball. ``` aws ecr get-login-password --region us-east-1 --profile LineajeGlosiRole | sudo docker login --username AWS --password-stdin .dkr.ecr.us-east-1.amazonaws.com ``` 2\. Pull the image in your host vm using below command ``` docker pull .dkr.ecr.us-east-1.amazonaws.com/glosi:latest ``` 3\. Use below command to deploy all services with docker-compose.yaml and image pulled in previous step. This will deploy all services in one single vm. ``` sudo docker compose -f up -d ```
### **Custom Deployment** The deployment can be customized. Here are the configuration files that can be customized. #### **ElasticSearch configuration** To start elastic search separately elasticsearch container can be spawned in separate vm and a port needs to be exposed to make it accessible through data service. With below configuration the container runs at recommended **9200** port, which can be changed as per requirement. ``` elasticsearch: ``` ```     image: .dkr.ecr.us-east-1.amazonaws.com/glosi:latest ``` ```     container_name: elasticsearch ``` ```     command: ["/bin/sh", "-c", "/usr/share/elasticsearch/bin/elasticsearch"] ``` ```     user: "1001:1001"  # Run as UID/GID defined in Dockerfile ``` ```     environment:      ``` ```       - discovery.type=single-node ``` ```       - ES_JAVA_OPTS=-Xms512m -Xmx512m ``` ```     volumes: ``` ```       - elasticsearch_data:/usr/share/elasticsearch/data ``` ```     ports: ``` ```       - "9200:9200" ``` ```     restart: no ``` ```     healthcheck: ``` ```       test: ["CMD-SHELL", "curl --silent --fail http://elasticsearch:9200/_cluster/health || exit 1"] ``` ```       interval: 5s ``` ```       retries: 10 ``` ```       timeout: 3s ``` #### **Data Service configuration** Data service is the data access API using FastAPI and connects to Elasticsearch. Converts client requests into Elasticsearch queries. This container runs on recommended port `8000` which can be customised with below configuration. This service also exposed a swagger docs endpoint at url: **http\://\:\/docs** ``` data-service: ``` ```     image: .dkr.ecr.us-east-1.amazonaws.com/glosi:latest ``` ```     container_name: data-service ``` ```     working_dir: /app/data-service ``` ```     command: ["gunicorn", "main:app", "-w", "4", "-k", "uvicorn.workers.UvicornWorker", "-b", "0.0.0.0:8000", "--timeout", "2", "--graceful-timeout", "15", "--keep-alive", "10"]    ``` ```     environment: ``` ```       - PATH=/app/data-service/venv/bin:$PATH ``` ```       - ELASTIC_AUTH_METHOD=NO_AUTH ``` ```       - OPENSEARCH_HOST=elasticsearch   ``` ```       - OPENSEARCH_PORT=9200 ``` ```       - USER_SSL=False ``` ```       - VERIFY_CERT=False ``` ```     depends_on: ``` ```       elasticsearch: ``` ```         condition: service_healthy ``` ```     ports: ``` ```       - "8000:8000" ``` ```     restart: always ``` #### **FrontEnd** The frontend using which user interact with this service through their browsers. It calls backend APIs and displays data visually. It runs on recommended 3000 port (`http://localhost:3000`) which can be customized with below container definition. ``` risklensui: ``` ```     image: .dkr.ecr.us-east-1.amazonaws.com/glosi:latest ``` ```     container_name: risklensui ``` ```     working_dir: /app/risklensui ``` ```     command: ["npm", "run", "start"]    ``` ```     ports: ``` ```       - "3000:3000" ``` ```     restart: always ``` #### **GLOSI Orchestration Service** Orchestration service handles workflows and connects UI actions with the data-service. Adds logic, routing, and validation. It runs on recommended 8500 port (**http\://\:\/docs**) which can be customised with below container definition. Here link to elastic search needs to be provided along with port as this service interacts with elastic search directly with data synchronisation. ``` orchestration-service: ``` ```     image: .dkr.ecr.us-east-1.amazonaws.com/glosi:latest ``` ```     container_name: orchestration-service ``` ```     working_dir: /app/glosi-orchestration-service ``` ```     command: ["gunicorn", "main:app", "-w", "4", "-k", "uvicorn.workers.UvicornWorker", "-b", "0.0.0.0:8500", "--timeout", "2", "--graceful-timeout", "15", "--keep-alive", "10"]    ``` ```     environment: ``` ```       - PATH=/app/glosi-orchestration-service/venv/bin:$PATH ``` ```       - DATA_SERVICE_HOST=data-service ``` ```       - DATA_SERVICE_PORT=8000 ``` ```       - ELASTIC_AUTH_METHOD=NO_AUTH ``` ```       - OPENSEARCH_HOST=elasticsearch   ``` ```       - OPENSEARCH_PORT=9200 ``` ```       - USER_SSL=False ``` ```       - VERIFY_CERT=False ``` ```       - OSS_TENANT_ID=vdna_994mgmr65tculnfy ``` ```     depends_on: ``` ```       - data-service ``` ```     ports: ``` ```       - "8500:8500" ``` ```     restart: always ``` ### **Data Persistence, Networking, and Volumes** **Volumes** ensure data inside containers isn’t lost when containers restart or rebuild. For instance, the `elasticsearch_data` volume stores all search and analytics data. Volumes used: · `elasticsearch_data`: Stores data generated by Elasticsearch post package and vulnerability index data synchronisation **Networking** Docker Compose automatically creates an internal bridge network. Containers communicate using service names. This abstraction avoids hardcoding IPs and simplifies scaling or relocating services. **Healthchecks** ensure each service is only marked healthy once it responds to a specific URL or command, improving reliability. ### **Service Interaction** Here’s how the services communicate: 1\. **Frontend Interaction** The user opens the UI at `http://localhost:3000`. This sends API requests (like search queries) to the orchestration-service at `http://localhost:8500`. 2\. **Orchestration-Service** Validates the request and forwards it to the appropriate backend (usually the data-service). 3\. **Data-Service** Receives requests from orchestration-service. Transforms them into Elasticsearch-compatible queries and sends them to Elasticsearch. 4\. **Elasticsearch** Performs search/indexing operations and returns results to the data-service. 5\. **Response Propagation** The data-service formats Elasticsearch results and returns them to orchestration-service, which may further process or log it. 6\. **Frontend Rendering** The ui receives the final result and displays it to the user. The architecture follows this loop: **UI ➝ Orchestration ➝ Data-service ➝ Elasticsearch ➝ Data-service ➝ Orchestration ➝ UI** Compose ensures services are started in this dependency chain, meaning each service waits for its required backend to be healthy before starting. ### **Access URLs** · Frontend: [http://](http://localhost:3000/)localhost:3000 · Data-Service Swagger Docs: [http://\:\/docs](http://localhost:8000/docs) i.e. [http://localhost:](http://localhost:3000)8000/docs · Glosi Orchestration Service Swagger Docs: [http://\:\/docs](http://localhost:8000/docs) i.e. [http://localhost:](http://localhost:3000)8500/docs ### GLOSI API reference This section provides a comprehensive overview of all available endpoints, request formats, and response structures. All APIs follow the OpenAPI 3.1.0 specification and are served via FastAPI. Swagger documentation is available at: **API Docs**: `http://:/docs` #### API Summary | Method | Route | Purpose | Input Type | Response Summary | | ------ | ------------------ | ------------------------------------- | ---------------------- | -------------------------------- | | POST | `/component` | Search component (package/image) info | `ComponentRequest` | Components list, totals, sample | | POST | `/search/packages` | Search packages | `SearchRequest` | List of packages with pagination | | POST | `/search/cves` | Search CVEs | `SearchRequest` | List of CVEs with pagination | | POST | `/search/images` | Search images | `SearchRequest` | List of images with pagination | | POST | `/catalog/search` | Search catalog entries | `CatalogSearchRequest` | Catalog search results | #### API Details **1. POST `/component`** **Description:** Retrieves detailed component information (package/image). **Request Body:** `ComponentRequest` ``` { ``` ```   "type": "package",  // or "image" ``` ```   "packageName": "example", ``` ```   "manager": "npm", ``` ```   "version": "1.0.0", ``` ```   "gos": ["gold", "premium"], ``` ```   "page": 0, ``` ```   "page_size": 20 ``` ``` } ``` **Behavior:** · If `type = package`, provides packages info · If `type = image`,provides images info **Response:** ``` { ``` ```   "status": true, ``` ```   "components": [...], ``` ```   "total": 120, ``` ```   "gos_total": 80, ``` ```   "sample_document": {...} ``` ``` } ``` **2. POST `/search/packages`** **Description:** Searches for packages with filters and overrides. **Request Body:** `SearchRequest` ``` { ``` ```   "searchQuery": "openssl", ``` ```   "gos_type": ["gold"], ``` ```   "page": 0, ``` ```   "page_size": 3, ``` ```   "overide_gos_filter": true ``` ``` } ``` **Response:** ``` { ``` ```   "status": true, ``` ```   "packages": [...], ``` ```   "pagination": { "packages": 30 } ``` ``` } ``` **3. POST `/search/cves`** **Description:** Searches for CVEs based on query, severity, and timeframe. **Request Body:** `SearchRequest` ``` { ``` ```   "searchQuery": "CVE-2023", ``` ```   "severity": ["high"], ``` ```   "timeframe": "2024-01-01", ``` ```   "page": 0, ``` ```   "page_size": 5 ``` ``` } ``` **Response:** ``` { ``` ```   "status": true, ``` ```   "cves": [...], ``` ```   "pagination": { "cves": 50 } ``` ``` } ``` **4. POST `/search/images`** **Description:** Searches for container images. **Request Body:** `SearchRequest` ``` { ``` ```   "searchQuery": "ubuntu", ``` ```   "gos_type": ["premium"], ``` ```   "overide_gos_filter": true ``` ``` } ``` **Response:** ``` { ``` ```   "status": true, ``` ```   "images": [...], ``` ```   "pagination": { "images": 17 } ``` ``` } ``` **5. POST `/catalog/search`** **Description:** Searches the full catalog with extended filters. **Request Body:** `CatalogSearchRequest` ``` { ``` ```   "searchQuery": "django", ``` ```   "managers": ["pypi"], ``` ```   "type": "package", ``` ```   "gos": ["gold"], ``` ```   "page": 0, ``` ```   "page_size": 10 ``` ``` } ``` **Response:** ``` { ``` ```   "status": true, ``` ```   "catalog": [...], ``` ```   "pagination": { "catalog": 55 } ``` ``` } ``` #### Schema Overview All endpoints expect input types as defined in OpenAPI schemas: · `ComponentRequest`: For `/component` · `SearchRequest`: For `/search/*` · `CatalogSearchRequest`: For catalog-based queries · `FixRequest`, `FixStatusRequest`, `CVERequest`, etc.: Used in fix and CVE resolution APIs Example of `SearchRequest` schema: ``` { ``` ```   "searchQuery": "string", ``` ```   "page": 0, ``` ```   "page_size": 3, ``` ```   "severity": ["low", "medium"], ``` ```   "gos_type": ["gold"], ``` ```   "overide_gos_filter": true ``` ``` } ``` **Notes** · All routes are served under FastAPI with auto-generated Swagger UI. · Swagger Docs available at `http://:/docs` · Supports flexible filtering, pagination, and override logic. ### **Maintenance and Troubleshooting** An admin panel is provided to check the status of data index status. **Maintenance Tips** · Clean up unused resources: `docker system prune -a` · Restart specific service: `docker-compose restart data-service` **Troubleshooting** · Check container status: `docker ps` · Access container shell: `docker exec -it /bin/sh` · Watch logs for errors. --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://docs.veedna.com/gold-open-source-gos/gold-catalog/container-images/glosi-global-lineaje-open-source-intelligence/user-guide.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.