Skip to content
Open
Show file tree
Hide file tree
Changes from 14 commits
Commits
Show all changes
53 commits
Select commit Hold shift + click to select a range
983e118
Dev environment with devcontainer
jboilard1994 Dec 7, 2024
d1b9906
Unix Filetype change
jboilard1994 Dec 7, 2024
4458d86
Proper installation and configuration of git in devcontainer
jboilard1994 Dec 7, 2024
c9a75d1
ADD jupyter notebook stripout on commit
jboilard1994 Dec 7, 2024
3e22092
test
jboilard1994 Dec 7, 2024
5fd976e
FIX precommit hooks
jboilard1994 Dec 7, 2024
d332bb8
Remove nbstripout git hook
jboilard1994 Dec 8, 2024
c2ef908
Reorder dockerfile layers so likely to change are earlier, maximizing…
jboilard1994 Dec 8, 2024
d9d3035
remove precommits in favour in devcontainer features
jboilard1994 Dec 8, 2024
7759ccf
Keep credentials folders
jboilard1994 Dec 8, 2024
f2e49a7
Normalize line endings using .gitattributes
jboilard1994 Dec 8, 2024
6713910
Normalize line endings in the master branch
jboilard1994 Dec 8, 2024
7b651ae
Normalize line endings using .gitattributes
jboilard1994 Dec 8, 2024
fc91fd3
Normalize line endings in the master branch
jboilard1994 Dec 8, 2024
a085eb1
REVERT to LF endings that were auto-converted
jboilard1994 Dec 8, 2024
c05bab8
REVERT LF to CRLF from pulling from windows host
jboilard1994 Dec 8, 2024
08b171a
amend commit
jboilard1994 Dec 8, 2024
4e9df95
Return to default version
jboilard1994 Dec 8, 2024
4c3491a
Remove now unused dev environment dockerfile and requirements
jboilard1994 Dec 8, 2024
9893f59
disable autocrlf when loading project in devcontainer in devcontainer…
jboilard1994 Dec 8, 2024
ae5c4f1
Better tracking of package roles
jboilard1994 Dec 8, 2024
dbe6a70
Dedicated mlflow container instead of executing the whole dev contain…
jboilard1994 Dec 8, 2024
6e8bc41
Change readme to be transparent with my changes and those I want to p…
jboilard1994 Dec 8, 2024
c0cb915
Merge branch 'master' into notebook-devcontainer
jboilard1994 Dec 8, 2024
aaa9c80
PUT CI in a straightforward yaml workflow
jboilard1994 Dec 8, 2024
716b7b1
trigger workflow
jboilard1994 Dec 8, 2024
830e80f
wrong workflow name
jboilard1994 Dec 8, 2024
cd50f61
remove old docker workflow
jboilard1994 Dec 8, 2024
7c3a74d
Create CI.yml
jboilard1994 Dec 8, 2024
e318ce5
Create CI.yml
jboilard1994 Dec 8, 2024
fec2e23
prevent all requirements from being installed on host github action r…
jboilard1994 Dec 8, 2024
c0725b0
Update CI.yml
jboilard1994 Dec 8, 2024
ea0abf7
Update CI.yml
jboilard1994 Dec 8, 2024
e8d7f9e
Update CI.yml
jboilard1994 Dec 8, 2024
f0ac567
Update CI.yml
jboilard1994 Dec 8, 2024
7eaecf4
Transfer flake8 configs so it is used during both github actions and …
jboilard1994 Dec 8, 2024
773442d
missing dev package
jboilard1994 Dec 8, 2024
2803d52
remove tracking on binaries
jboilard1994 Dec 8, 2024
9f2206f
Add tracking of the artifacts and features folder
jboilard1994 Dec 8, 2024
31ae321
ADD default features for model
jboilard1994 Dec 8, 2024
d2ec482
Update readme with devcontainer local changes
jboilard1994 Dec 9, 2024
6eb7f6d
adjust to new venv workdir relative path
jboilard1994 Dec 12, 2024
2c3029c
Adjust to venv relative path + remove manually managed versions
jboilard1994 Dec 12, 2024
457af1e
enable black formatting and QOL notebook features
jboilard1994 Dec 12, 2024
9bb37c2
Wrong default config
jboilard1994 Dec 12, 2024
cb41fcf
Tests now pass locally
jboilard1994 Dec 12, 2024
4c289ed
remove tracking of html files
jboilard1994 Dec 12, 2024
af6c31a
remve .gitinclude file since folder is deleted if cloud configs setupped
jboilard1994 Dec 12, 2024
6a1829c
Upgraded workflow which registers best performing models at each expe…
jboilard1994 Dec 13, 2024
1955ec0
FIX flake8 verification
jboilard1994 Dec 13, 2024
76eee8f
ADD a default set of features and of data to allow tests to pass with…
jboilard1994 Dec 13, 2024
2543da6
Nevermind that last commit ¯\_(ツ)_/¯
jboilard1994 Dec 13, 2024
5e7548b
Merge branch 'master' into notebook-devcontainer
jboilard1994 Dec 13, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .devcontainer/.env
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
GIT_USERNAME="jboilard1994"
GIT_EMAIL="jboilard1994@gmail.com"
2 changes: 2 additions & 0 deletions .devcontainer/.env.template
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
GIT_USERNAME="FIRST_NAME LAST_NAME"
GIT_EMAIL="MY_NAME@example.com"
Empty file.
79 changes: 79 additions & 0 deletions .devcontainer/devcontainer.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
// For format details, see https://aka.ms/devcontainer.json. For config options, see the
// README at: https://github.com/devcontainers/templates/tree/main/src/anaconda-postgres
{
"name": "${localWorkspaceFolderBasename}-devcontainer",
"dockerComposeFile": [
"../docker-compose.yaml",
"docker-compose.dev.override.yaml"
],
"hostRequirements": {
"gpu": "optional"
},
"service": "app",
"workspaceFolder": "/workspace",

// Container customizations
"customizations": {
"vscode": {
// VS Code-specific settings
"settings": {
// Python interpreter and module path
"python.pythonPath": "/workspace/src",
"python.defaultInterpreterPath": "/opt/venv/_dev/bin",

// Formatting and quality of life
"editor.formatOnSave": true,
"editor.codeActionsOnSave": {
"source.organizeImports": "explicit"
},
"isort.args": ["--profile", "black"],
"python.analysis.typeCheckingMode": "basic",
"python.languageServer": "Pylance",
"editor.defaultFormatter": "ms-python.black-formatter",

// Linting
"python.linting.enabled": true,
"python.linting.flake8Enabled": true,
"python.linting.flake8Args": ["--max-line-length=120"],
"python.linting.mypyEnabled": true,
"mypy-type-checker.args": ["--ignore-missing-imports", "--disallow-untyped-defs", "--disallow-incomplete-defs", "--cache-dir", ".mypy_cache"],
// Unit testing
"python.testing.pytestEnabled": true,
"python.testing.unittestEnabled": false,
"python.testing.pytestPath": "pytest",
"python.testing.pytestArgs": [
"tests",
"--disable-pytest-warnings"
],

// Docker settings
"docker.enableTelemetry": false,
"docker.languageserver.diagnostics.enabled": true
},

// Extensions to install in the container
"extensions": [
"ms-vscode-remote.remote-containers",
"ms-python.python",
"ms-python.vscode-pylance",
"ms-python.flake8",
"ms-python.mypy-type-checker",
"ms-python.black-formatter",
"ms-python.isort",
"ms-azuretools.vscode-docker",
"eamodio.gitlens",
"ms-toolsai.jupyter",
"yzhang.markdown-all-in-one",
"DavidAnson.vscode-markdownlint"
]
}
},

// Commands to run during setup
"postCreateCommand": "chmod +x /workspace/.devcontainer/setup.sh && /workspace/.devcontainer/setup.sh",

// Features to add to the container
"features": {
"git": "latest"
}
}
10 changes: 10 additions & 0 deletions .devcontainer/docker-compose.dev.override.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
services:
app:
command: ["/bin/bash"]
env_file:
- .devcontainer/.env
tty: true

# setting repo root folder as container workspace
volumes:
- ./:/workspace
3 changes: 3 additions & 0 deletions .devcontainer/noop.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
This file copied into the container along with environment.yml* from the parent
folder. This file is included to prevents the Dockerfile COPY instruction from
failing if no environment.yml is found.
7 changes: 7 additions & 0 deletions .devcontainer/setup.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
#!/bin/sh

# git config
git config --global user.name $GIT_USERNAME
git config --global user.email $GIT_EMAIL
git config --global credential.useHttpPath true
git config --global core.filemode false
4 changes: 4 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
* text eol=lf
images/* -text
*.pkl -text
*.png -text
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
.devcontainer/.env
.devcontainer/credentials
logs/
ipynb_checkpoints/
mlruns
Expand Down
30 changes: 0 additions & 30 deletions .pre-commit-config.yaml

This file was deleted.

18 changes: 9 additions & 9 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,8 +1,14 @@
# using the lastest version of Ubuntu 22.04 as a base for the Docker image
FROM ubuntu:22.04
FROM python:3.9-slim
ENV VENV_DIR=/opt/venv/_dev
ENV PATH=$VENV_DIR/bin:$PATH
SHELL ["/bin/bash", "-c"]

# installing Python and Unzip
RUN apt-get update && apt-get install -y python3.10 python3.10-venv python3.10-dev python3-pip libgomp1
COPY ./requirements.txt /.
RUN python -m venv $VENV_DIR \
&& $VENV_DIR/bin/pip install --upgrade pip \
&& source $VENV_DIR/bin/activate \
&& pip install -r /requirements.txt

# creating the root folder
RUN mkdir -p /e2e-mlops-project
Expand All @@ -18,9 +24,3 @@ WORKDIR /e2e-mlops-project
RUN rm -r /e2e-mlops-project/notebooks/

COPY ./notebooks/VERSION /e2e-mlops-project/notebooks/VERSION

# updating pip
RUN pip install --no-cache-dir -U pip

# installing requirements
RUN pip install -r requirements.txt
Loading