Skip to content

✨ Dataservice Models - PCGC#19

Open
Christina-J-Diaz wants to merge 11 commits intomainfrom
feature/cd/dataservice-model
Open

✨ Dataservice Models - PCGC#19
Christina-J-Diaz wants to merge 11 commits intomainfrom
feature/cd/dataservice-model

Conversation

@Christina-J-Diaz
Copy link
Contributor

@Christina-J-Diaz Christina-J-Diaz commented Feb 13, 2026

♻️ Create model for PCGC and DS studies

Re-do of #13 :

This PR builds upon a previous KF study test model (see #8). This model uses a more complex study that covers more DS entities than the previous example, so that we can model dataservice entirely.

Also included, is adding a DAG for this study.

Some key decisions/points:

This model focuses on mirroring dataservice entities into the model. Therefore, transformations are minimal. The key focus is getting the structure and flow right which we can then adjust later
The stable and access models are also minimal as they were not the focus. These models are filled in to give a proof of concept, but will need to be updated to match the final access model
Comments have been added to fields/entites that could be improved or removed entirely from the model, as they will not be needed in any upstream models
Visibility has not been applied but eventually should - as all dataservice records are not eligible to flow into access models. This will need to be further discussed.
All tables have been named using this format: program_study_stage_a . In this case, the idea is this model would eventually be used for all existing dataservice studies. Rather than one model per study, there would be one model for all dataservice studies. With that in mind, I've named models as such: kf_ds_src_, kf_ds_int_, kf_ds_stable_ . With this naming, we can use this model as a base and add more studies without needing to make major changes to table names/docs.
Documentation has been updated to fit this model - it does deviate from the documentation from the initial test study

Closes (issue identifier)

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality
    to not work as expected)
  • This change requires a documentation update

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide
instructions so we can reproduce. Please also list any relevant details for
your test configuration

  • successfully run dbt locally
  • review dev schema for expected outputs
  • locally review docs for accuracy

Test Configuration:

  • Environment:
  • Test files:

Checklist

Please check all of the items below before merging this pull request. If an
item in the list below does not need to be completed, please indicate the
reason why.

  • I ensured that all tables have proper source and ref definitions
  • I Defined {{ config(schema=[schema name], tags = [list, of, tags]) }} at
    the start of each script
  • I have performed a self-review of my own code
  • I have checked my code and corrected any misspellings
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have added tests that prove my fix is effective or that my feature
    works
  • My code follows the style guidelines of this project
  • My changes generate no new warnings
  • New and existing unit tests pass locally with my changes
  • I have committed any related changes to the PR
  • Run sqlfluff fix on tables that are created or modified in this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant