Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: refactoring of the codebase is needed #99

Closed
masterchief164 opened this issue May 12, 2023 · 4 comments
Closed

chore: refactoring of the codebase is needed #99

masterchief164 opened this issue May 12, 2023 · 4 comments

Comments

@masterchief164
Copy link
Contributor

No description provided.

@adamjonas
Copy link
Member

Can you describe what you believe needs to be done ?

@masterchief164
Copy link
Contributor Author

The inclusion of multiple transcribers with many options like diarization, summary creation, and topic detection has made the code a lot bloated. There are a lot of if else ladders that can be simplified. Also, the methods do not follow the SOLID principles. I'd like to address those issues and thus make the code a bit more maintanable.

@SarcasticNastik
Copy link
Collaborator

SarcasticNastik commented Jun 6, 2023

Refactor Strategy

Here, I outline a general idea for refactoring the codebase using an object-oriented approach.

Application Workflow

Pre-process source files

  • Given source: Video|Audio|Yt-link, conditionally pre-process ( download and convert) the source files.

Current approach for detecting and specifying the source type and the corresponding processing is manual using source_type.
Automatic detection of file type and further processing should be achieved using either:

  • Pattern matching using match (supported by python versions $\ge$ 10).
  • Conditional matching.

Transcription

  • Transcribe the audio input to text using either deepgram or whisper.

The corresponding codebase can be easily ported.

Post-process generated files

  • Store the generated transcription from the previous section.
  • Optionally, generate a PR for the transcription.
  • Optionally, upload model outputs to AWS S3.

Again, all the corresponding functionalities can be easily ported.

App class

Methods

Corresponding to the previous section.

  1. pre_process (needs a better name)

    • detect_source_type
    • download (conditionally)
    • convert (conditionally)
  2. transcribe

  3. post_process (needs a better name)

    • write_transcription_to_md
    • create_pr (conditionally)
    • upload_to_s3 (conditionally)
  4. process: Complete workflow. (end-user API)

@kouloumos
Copy link
Member

Codebase has been refactored using an object-oriented approach into a structured four-stage process. Most of the work has been done with #118 and 2764a7f, with additional changes in miscellaneous commits that can be found in the git history.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants