Remove airflow dependencies from handlers

[helen-m-lin]

User story

As a developer, I want to remove the Airflow dependencies from the handlers so I can minimize the runtime environment and better organize the Airflow and orchestration workflow.

Acceptance criteria

• Refactor the handlers so that the expected Airflow Variables/Connections are read from environment variables.
• Update the alert_handler to use a different method to send emails.
• Remove dependencies on Airflow SSHHook, BaseHook
• Remove airflow dependencies from the pyproject.toml file.

Sprint Ready Checklist

• 1. Acceptance criteria defined
• 2. Team understands acceptance criteria
• 3. Team has defined solution / steps to satisfy acceptance criteria
• 4. Acceptance criteria is verifiable / testable
• 5. External / 3rd Party dependencies identified
• 6. Ticket is prioritized and sized

Notes

We need to refactor to remove these dependencies:

from airflow.models import Connection, Variable # log_handler, slurm_v2_handler
from airflow.models import Variable # task_handler
from airflow.utils.email import send_email # alert_handler
from airflow.providers.ssh.hooks.ssh import SSHHook # hpc_handler, slurm_v2_handler
from airflow.hooks.base import BaseHook # log_handler, slurm_v2_handler