Merge pull request #7 from safedep/develop

OpenSearch Indexing

Author: abhisek

.github/workflows/codeql.yml

@@ -1,12 +1,8 @@
 name: "CodeQL"
 
 on:
-  push:
-    branches: [ "main" ]
   pull_request:
     branches: [ "main" ]
-  schedule:
-    - cron: '55 23 * * 0'
 
 jobs:
   analyze:

.github/workflows/dependency-review.yml

@@ -18,3 +18,5 @@ jobs:
         uses: actions/checkout@v3
       - name: 'Dependency Review'
         uses: actions/dependency-review-action@v2
+        with:
+          fail-on-severity: high

README.md

@@ -1,23 +1,8 @@
-# Supply Chain Security Gateway
-
-A reference architecture and **<ins>proof of concept implementation</ins>** of a supply chain security gateway with the goal of enforcing sane security policies to an organization's consumption of 3rd party software (dependencies) in its own products.
-
-- [Supply Chain Security Gateway](#supply-chain-security-gateway)
-  - [TL;DR](#tldr)
-  - [Architecture](#architecture)
-    - [Data Plane Flow](#data-plane-flow)
-  - [Usage](#usage)
-    - [Configuring Upstream and Routes](#configuring-upstream-and-routes)
-    - [Configuring Environments](#configuring-environments)
-    - [Authentication](#authentication)
-      - [Ingress Authentication](#ingress-authentication)
-        - [Basic Authentication](#basic-authentication)
-  - [Development](#development)
-    - [PDP Development](#pdp-development)
-    - [Policy Development](#policy-development)
-    - [Tap Development](#tap-development)
-    - [Debug NATS Messaging](#debug-nats-messaging)
-  - [Contribution](#contribution)
+# SafeDep: Security Gateway
+
+Security gateway can be used to establish policy control over consumption of 3rd party software components. [https://safedep.io](https://safedep.io) to get started with usage.
+
+> This README is for Gateway developers. Refer to [https://safedep.io](https://safedep.io) for usage instructions for various environments and use-cases.
 
 ## TL;DR
 
@@ -54,7 +39,7 @@ Verify all the services are active
 docker-compose ps
 ```
 
-Use the gateway using a `demo-client`
+Use the gateway using a [demo-client](https://github.com/safedep/demo-client-java)
 
 ```bash
 cd demo-clients/java-gradle && ./gradlew assemble --refresh-dependencies
@@ -69,7 +54,7 @@ docker-compose logs pdp
 
 The `gradle` build should fail with an error message indicating a dependency was blocked by the gateway.
 
-```
+```bash
 > Could not resolve all files for configuration ':app:compileClasspath'.
    > Could not resolve org.apache.logging.log4j:log4j:2.16.0.
      Required by:
@@ -81,7 +66,7 @@ The `gradle` build should fail with an error message indicating a dependency was
 
 > Refer to `policies/example.rego` for the policy that blocked this artefact
 
-Edit `config/global.yml` and set `pdp.monitor_mode=true` to enable only monitoring and disable policy enforcement. Restart the containers for the changes to take effect.
+Edit `config/gateway.json` and set `pdp.monitor_mode=true` to enable only monitoring and disable policy enforcement. Restart the containers for the changes to take effect.
 
 ```bash
 docker-compose up --force-recreate --remove-orphans --build -d
@@ -109,114 +94,10 @@ If you are developing on any of the service and want to force re-create the cont
 docker-compose up --force-recreate --remove-orphans --build -d
 ```
 
-### Configuring Upstream and Routes
-
-The configuration plane is currently half baked. It needs a tool and a single source of truth to generate configuration for Envoy and Gateway. For now, look at:
-
-1. `config/global.yml`
-2. `config/envoy.yml`
-
-> The route definitions in `envoy.yml` must match the path patterns in `global.yml`
-
-### Configuring Environments
-
-To use the gateway in a CI or developer local environment, package managers need to be configured to use the gateway URL and credentials as repository.
-
-[PacMan](pacman/README.md) makes it easy to automatically configure an environment to use the gateway for downloading 3rd party dependencies.
-
-### Authentication
-
-There are two authentication points:
-
-1. Ingress
-2. Egress
-
-Ingress authentication is for incoming requests to the gateway and can be used to identify who is accessing the gateway.
-
-Egress authentication is for upstream repositories, especially the ones that need authentication e.g. CodeArtifact, JFrog, Nexus etc.
-
-- [Ingress Gateway Authentication](docs/Gateway-Authentication.md)
-
-#### Ingress Authentication
-
-##### Basic Authentication
-
-Use `htpasswd` to add users:
-
-```bash
-htpasswd -nbB user1 password1 >> ./config/gateway-auth-basic
-```
-
-Enable authentication for upstream in `config/global.yml`
-
-## Development
-
-### PDP Development
+## Developer Documentation
 
-Build and run the PDP using:
-
-```bash
-cd services && make
-GLOBAL_CONFIG_PATH=../config/global.yml PDP_POLICY_PATH=../policies ./out/pdp-server
-```
-
-PDP listens on `0.0.0.0:9000`. To use the host instance of PDP, edit `config/envoy.yml` and set the address of the `ExtAuthZ` plugin to your host network address.
-
-### Policy Development
-
-Policies are written in [Rego](https://www.openpolicyagent.org/docs/latest/policy-language/) and evaluated with [Open Policy Agent](https://www.openpolicyagent.org/docs/latest/integration/#integrating-with-the-go-api)
-
-To run policy test cases:
-
-```bash
-cd policies && make test
-```
-
-* Refer to `policies/example.rego` for policy example
-* Policies are load from `./policies` directory
-
-### Tap Development
-
-The *Tap Service* is integrated as a Envoy [ExtProc](https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/ext_proc_filter) filter. This means, it has greater control over Envoy's request processing life-cycle and can make changes if required.
-
-Currently, it is used for publishing events for data collection only but in future may be extended to support other use-cases. Tap service internally implements a handler chain to delegate an Envoy event to its internal handlers. Example:
-
-```go
-tapService, err := tap.NewTapService(config, []tap.TapHandlerRegistration{
-  tap.NewTapEventPublisherRegistration(),
-})
-```
-
-To build and use from host:
-
-```bash
-cd services && make
-GLOBAL_CONFIG_PATH=../config/global.yml ./out/tap-server
-```
-
-> To use Tap service from host, edit `envoy.yml` and change address of `ext-proc-tap` cluster.
-
-### Debug NATS Messaging
-
-Start a docker container with `nats` client
-
-```bash
-docker run --rm -it \
-   --network supply-chain-security-gateway_default \
-   -v `pwd`:/workspace \
-   synadia/nats-box
-```
-
-Subscribe to a subject and receive messages
-
-```bash
-GODEBUG=x509ignoreCN=0 nats sub \
-   --tlscert=/workspace/pki/tap/server.crt \
-   --tlskey=/workspace/pki/tap/server.key \
-   --tlsca=/workspace/pki/root.crt \
-   --server=tls://nats-server:4222 \
-   com.msg.event.upstream.request
-```
+* [Authentication](docs/Authentication.md)
+* [Service Development](docs/Development.md)
 
 ## Contribution
 

bootstrap.sh

@@ -43,5 +43,6 @@ MYSQL_DCS_USER=root
 MYSQL_DCS_PASSWORD=$mysql_root_pass
 
 KAFKA_PONGO_HOST=127.0.0.1
+OPENSEARCH_HOST=172.17.0.1
 _EOF
 fi