Skip to content

Release v1.19.0 #316

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
helen-m-lin merged 4 commits into from
Sep 29, 2025
Merged

Release v1.19.0 #316

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
1ba75fb
Merge pull request #311 from AllenNeuralDynamics/main
jtyoung84 Sep 22, 2025
e94260d
feat: removes getCurrentVersion function and hardcodes modal version …
johendrickson Sep 24, 2025
75f023d
feat: view full job JSON of uploads in UI before submission (#314)
johendrickson Sep 29, 2025
90b77ed
ci: version bump
johendrickson Sep 29, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion src/aind_data_transfer_service/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

import os

__version__ = "1.18.2"
__version__ = "1.19.0"

# Global constants
OPEN_DATA_BUCKET_NAME = os.getenv("OPEN_DATA_BUCKET_NAME", "open")
Expand Down
3 changes: 1 addition & 2 deletions src/aind_data_transfer_service/server.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -763,8 +763,7 @@ async def job_params(request: Request):
"project_names_url": os.getenv(
"AIND_METADATA_SERVICE_PROJECT_NAMES_URL"
),
"versions": ["v1", "v2"],
"default_version": "v1",
"default_version": "v2",
"modalities": JobParamInfo._MODALITIES_LIST,
"modality_tasks": JobParamInfo._MODALITY_TASKS,
}
Expand Down
81 changes: 63 additions & 18 deletions src/aind_data_transfer_service/templates/index.html
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,10 @@
<!DOCTYPE html>
<html>
<head>
<link href="https://cdn.jsdelivr.net/npm/bootstrap@5.3.3/dist/css/bootstrap.min.css" rel="stylesheet">
<link href="https://cdnjs.cloudflare.com/ajax/libs/bootstrap-icons/1.8.1/font/bootstrap-icons.min.css" rel="stylesheet">
<script src="https://ajax.googleapis.com/ajax/libs/jquery/3.6.4/jquery.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/bootstrap@5.3.3/dist/js/bootstrap.bundle.min.js"></script>
<meta charset="UTF-8">
<title>{% block title %} {% endblock %} AIND Data Transfer Service</title>
<style>
Expand All @@ -10,7 +13,9 @@
font-family: arial, sans-serif;
}
fieldset {
display:inline
display:inline;
border:1px solid #dddddd;
padding: 10px 15px;
}
table {
border-collapse: collapse;
Expand All @@ -25,7 +30,7 @@
background-color: #E8EAF6;
color: #3F51B5;
}
#submit {
#submit {
border: none;
border-radius: 6px;
color: #ffffff;
Expand Down Expand Up @@ -56,7 +61,7 @@
<div>
<p1>For instructions on how to submit jobs click on the Help tab</p1>
</div>
<h2>Submit Jobs</h2>
<h4 class="my-3 fw-bold">Submit Jobs</h4>
<div>
<fieldset>
<legend>Mail Notifications (optional)</legend><br>
Expand Down Expand Up @@ -85,6 +90,19 @@ <h2>Submit Jobs</h2>
<button id="submit" type="button" onclick="submitJobs()">Submit</button>
<div id="message"></div><br>
<div id="response"></div>
<div class="modal fade" id="json-modal" tabindex="-1" aria-labelledby="json-modal-label">
<div class="modal-dialog modal-lg">
<div class="modal-content">
<div class="modal-header p-2">
<h5 class="modal-title" id="json-modal-label">json</h5>
<button type="button" class="btn-close" data-bs-dismiss="modal" aria-label="Close"></button>
</div>
<div class="modal-body p-2">
<pre id="json-content" style="white-space: pre-wrap; font-size: small;"></pre>
</div>
</div>
</div>
</div>
<script>
var jobs = []
var parsing_errors = []
Expand All @@ -106,7 +124,7 @@ <h2>Submit Jobs</h2>
$("#message").addClass("error");
}
};
addTableRow = function(data, table, tr, td, isHeader) {
addTableRow = function(data, table, tr, td, isHeader, job = null) {
tr = document.createElement('tr');
for (var d of data) {
td = document.createElement(isHeader ? 'th' : 'td');
Expand All @@ -116,13 +134,29 @@ <h2>Submit Jobs</h2>
}
tr.append(td);
}
if (job && !isHeader) {
const tdModal = document.createElement(isHeader ? 'th' : 'td');
const openModalButton = document.createElement("button");
openModalButton.type = "button";
openModalButton.className = "btn btn-outline-primary btn-sm";
openModalButton.dataset.bsToggle = "modal";
openModalButton.dataset.bsTarget = "#json-modal";
openModalButton.innerHTML = `<i class="bi bi-box-arrow-up-right"></i>`;
openModalButton.dataset.jobJSON = JSON.stringify(job, null, 3);
tdModal.append(openModalButton);
const modalitiesLength = job.tasks?.modality_transformation_settings
? Object.keys(job.tasks.modality_transformation_settings).length
: 1;
tdModal.setAttribute("rowspan", modalitiesLength);
tr.append(tdModal);
}
table.appendChild(tr);
};
validateJobs = function(fileElement) {
if (fileElement.files.length != 1) {
// File attach was cancelled by user
return;
}
}
if (![".csv", ".xlsx"].some(ext => fileElement.files[0].name.endsWith(ext))) {
fileElement.value = null;
alert("Invalid file type. Please attach a .csv or .xlsx file.");
Expand All @@ -143,11 +177,11 @@ <h2>Submit Jobs</h2>
success: function(data) {
setMessage(msgTypes.validateSuccess);
jobs = data["data"]["jobs"];
parsing_errors = []
parsing_errors = [];
let jobsLength = jobs.length;
var table = document.createElement('table'), tr, td, row;
addTableRow(
[ "job_type", "project_name", "s3_bucket", "platform", "subject_id", "acq_datetime", "metadata_dir", "modality", "modality.input_source" ],
["job_type", "project_name", "s3_bucket", "platform", "subject_id", "acq_datetime", "metadata_dir", "modality", "modality.input_source", "json"],
table, tr, td, true
);
for (row = 0; row < jobsLength; row++) {
Expand All @@ -157,10 +191,10 @@ <h2>Submit Jobs</h2>
modalities = Object.entries(modalities).map(([key, value]) => ({
abbreviation: key,
input_source: value.job_settings?.input_source
}))
}));
}
let modalitiesLength = modalities ? modalities.length: 0;
let metadata_dir = job.tasks?.gather_preliminary_metadata?.job_settings?.metadata_dir
let metadata_dir = job.tasks?.gather_preliminary_metadata?.job_settings?.metadata_dir;
addTableRow(
[
{ value: job.job_type, rowspan: modalitiesLength },
Expand All @@ -172,10 +206,10 @@ <h2>Submit Jobs</h2>
{ value: metadata_dir ?? "", rowspan: modalitiesLength },
modalities ? modalities[0].abbreviation : "",
modalities ? modalities[0].input_source : ""
], table, tr, td, false
], table, tr, td, false, job
);
for (mRow = 1; mRow < modalitiesLength; mRow++) {
let modality = modalities[mRow]
for (let mRow = 1; mRow < modalitiesLength; mRow++) {
let modality = modalities[mRow];
addTableRow(
[ modality.abbreviation, modality.input_source ],
table, tr, td, false
Expand All @@ -185,12 +219,10 @@ <h2>Submit Jobs</h2>
$("#response").html(table);
},
error: function(data) {
jobs = []
parsing_errors = data.responseJSON["data"]["errors"]
jobs = [];
parsing_errors = data.responseJSON["data"]["errors"];
setMessage(msgTypes.validateError);
$("#response").html(parsing_errors.map((err) => {
return `<li>${err}</li>`
}));
$("#response").html(parsing_errors.map(err => `<li>${err}</li>`));
}
});
};
Expand Down Expand Up @@ -253,6 +285,19 @@ <h2>Submit Jobs</h2>
return;
}
};
document.addEventListener("DOMContentLoaded", () => {
var jsonModal = document.getElementById('json-modal');
jsonModal.addEventListener('show.bs.modal', function(event) {
var button = event.relatedTarget;
if (!button) return;
var jobJSON = button.dataset.jobJSON;
var jsonContentDiv = document.getElementById("json-content");
jsonContentDiv.textContent = jobJSON;
});
jsonModal.addEventListener('hidden.bs.modal', function() {
document.getElementById("json-content").textContent = '';
});
});
</script>
</body>
</html>
</html>
39 changes: 9 additions & 30 deletions src/aind_data_transfer_service/templates/job_params.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,17 +44,6 @@
</nav>
<div class="content">
<h4 class="mb-2">
<!-- dropdown to switch version -->
<div id="version-dropdown" class="btn-group mb-2">
<button id="version-button" type="button" class="btn btn-primary dropdown-toggle" data-bs-toggle="dropdown">
{{default_version}}
</button>
<ul class="dropdown-menu">
{% for v in versions %}
<button class="dropdown-item" type="button">{{ v }}</button>
{% endfor %}
</ul>
</div>
<span>Job Parameters</span>
</h4>
<!-- button and modal for adding new parameters-->
Expand Down Expand Up @@ -135,6 +124,7 @@ <h5 class="modal-title" id="param-modal-label"></h5>
</div>
<script>
const MODALITY_TASKS = {{ modality_tasks | tojson }}; // from Jinja context
const DEFAULT_VERSION = "{{ default_version }}";
const MODAL_ID = 'param-modal';
$(document).ready(function() {
createJobParamsTable();
Expand All @@ -148,7 +138,7 @@ <h5 class="modal-title" id="param-modal-label"></h5>
modal.data('bs-action', action); // save action type
const isNew = action === 'new';
modal.find(`#${MODAL_ID}-label`).text(isNew ? 'Add New Parameter' : eventTarget.data('bs-param-name'));
modal.find(`#${MODAL_ID}-version`).text(getCurrentVersion()).toggle(isNew);
modal.find(`#${MODAL_ID}-version`).text(DEFAULT_VERSION).toggle(isNew);
modal.find(`#${MODAL_ID}-dropdowns`).toggle(isNew);
if (isNew) {
['Job Type', 'Task ID'].forEach(field => {
Expand All @@ -173,7 +163,7 @@ <h5 class="modal-title" id="param-modal-label"></h5>
const action = modal.data('bs-action');
let url;
if (action === "new") {
const version = getCurrentVersion();
const version = DEFAULT_VERSION;
const jobType = getValidatedInputValue('Job Type', modal);
const taskId = getValidatedInputValue('Task ID', modal);
const modality = MODALITY_TASKS.includes(taskId) ? getValidatedInputValue('Modality', modal) : null;
Expand All @@ -197,14 +187,6 @@ <h5 class="modal-title" id="param-modal-label"></h5>
$(`#${MODAL_ID}-modality-select`).on('change', function() {
handleExistingParam();
});
// Event listener for version dropdown
$('#version-dropdown .dropdown-item').on('click', function() {
var version = $(this).text();
if (version != getCurrentVersion()) {
$('#version-button').text(version);
$('#job-params-table').DataTable().ajax.url(`/api/${version}/parameters`).load();
}
});
});
// Create DataTable for job params table
function createJobParamsTable() {
Expand Down Expand Up @@ -259,12 +241,12 @@ <h5 class="modal-title" id="param-modal-label"></h5>
function renderParameterButton(data, type, row) {
if (type == 'display') {
return (
`<button type="button" class="btn btn-link btn-sm"
data-bs-toggle="modal"
`<button type="button" class="btn btn-link btn-sm"
data-bs-toggle="modal"
data-bs-target="#${MODAL_ID}"
data-bs-action="edit"
data-bs-param-name=${data}
data-bs-job-type=${row.job_type}
data-bs-param-name=${data}
data-bs-job-type=${row.job_type}
data-bs-task-id=${row.task_id}
data-bs-modality=${row.modality}
>${data}
Expand All @@ -280,7 +262,7 @@ <h5 class="modal-title" id="param-modal-label"></h5>
}
function getParamUrlFromParamName(paramName) {
const match = paramName.match(/job_types\/(v\d+)?\/?(.*)/);
const version = match && match[1] ? match[1] : 'v1';
const version = match && match[1] ? match[1] : DEFAULT_VERSION;
const cleanedParamName = match ? match[2] : paramName;
return `/api/${version}/parameters/job_types/${cleanedParamName}`;
}
Expand Down Expand Up @@ -338,7 +320,7 @@ <h5 class="modal-title" id="param-modal-label"></h5>
function handleExistingParam() {
let exists = false;
const modal = $(`#${MODAL_ID}`);
const version = getCurrentVersion();
const version = DEFAULT_VERSION
const jobType = getInputValues('Job Type', modal).input;
const taskId = getInputValues('Task ID', modal).input;
const modality = MODALITY_TASKS.includes(taskId) ? getInputValues('Modality', modal).input : null;
Expand Down Expand Up @@ -368,9 +350,6 @@ <h5 class="modal-title" id="param-modal-label"></h5>
}
}
// Helper methods to get current values
function getCurrentVersion() {
return $('#version-button').text().trim();
}
function getUniqueColumnValues(columnTitle) {
return Array.from(new Set($('#job-params-table').DataTable().column(`${columnTitle}:title`).data().toArray()));
}
Expand Down