SDK Installer test on all platforms #156

tijcolem · 2021-10-29T02:12:14Z

Pull request overview

This adds in a new GitHub actions to run regressions tests on all platforms using both cli and ruby bindings. I had to create a fork in order to test the GitHub actions as it needs to be on develop (at least I think it does). It's setup pretty much the same as the Test SDK Installer except the branch commits. I suppose this could be added but I think we can use the Test SDK Installer workflow to commit the test results.

This one is still running as of now. https://github.com/tijcolem/OpenStudio-resources/actions/runs/1397133553
Just waiting to make sure all steps work but I think this is close.

Expand platforms

jmarrec

Couple of small adjustments needed but it's 95% there! Nice!

jmarrec · 2021-10-29T16:42:05Z

ci/install-windows.qs

@@ -0,0 +1,41 @@
+function Controller () {


Same as the mac one really you need a single install-script.qs

Fair point. this is done

jmarrec · 2021-10-29T16:43:13Z

.github/workflows/manual_installer_all_platforms.yml

+      os_installer_ubuntu_1804:
+        description: 'Ubuntu 1804 (.deb) URL'
+        required: true
+        default: 'https://github.com/NREL/OpenStudio/releases/download/v3.2.1/OpenStudio-3.2.1%2Bbdbdbc9da6-Ubuntu-18.04.deb'


This is tedious to launch. We can infer the installer links for all platforms from a single one supplied.

I suppose OS builds will always have the base url.

Well I take that back. Windows is a little odd as I push out both unsigned and signed installers so s3 on nightly build that we'll certainly want to test and hook into this action.

e.g. http://openstudio-ci-builds.s3-website-us-west-2.amazonaws.com/?prefix=3.3.0-rc2/
signed: http://openstudio-ci-builds.s3-website-us-west-2.amazonaws.com/?prefix=3.3.0-rc2/signed/

Even though it's a bit tedious to enter 4 links there may be cases it's needed.

So you went back on this idea of having 4 links?

It's now just base url along with build sha, version.

jmarrec · 2021-10-29T16:51:14Z

.github/workflows/manual_installer_all_platforms.yml

+        ruby -v
+        MT_CPU=$(nproc) ruby model_tests.rb
+
+    - name: Generate HTML and heatmap


You probably mean to run this step twice, after each run of model_tests.rb (cli + run)

You probably want to run with CUSTOMTAG=cli and CUSTOMTAG=ruby

And call process_results.py with --tagged or --all

Yep, that's a good idea to tag them as cli and ruby

Do we really want to run model_tests.rb with both openstudio (CLI) and system ruby? That's gonna take a long time. The fact that we would want to do it on Windows I can see, but linux/mac I'm not sure.

I plan on these running nightly so I don't mind if they take a while to run. If we want to hook this into PRs on OpenStudio I think we should just run cli and maybe just one platform but we can use the other workflow for that. Maybe rename the workflow to _installer or something along those lines.

jmarrec · 2021-10-29T16:51:33Z

.github/workflows/manual_installer_all_platforms.yml

+
+jobs:
+  installer_ubuntu_1804:
+    runs-on: ubuntu-18.04


There's code that's duplicated for each job, but using a matrix and a bunch of if platform (in bash or conditionally run some steps) is probably not going to simplify much so I'm fine with it

Yeah, I thought of this. My understanding is the matrix works great for testing things such as multiples versions of python, not sure how if matrix works for vm instances.

jmarrec · 2022-04-13T19:45:57Z

.github/workflows/manual_installer_all_platforms.yml

      run: |
        set -x
-        echo "Installer link: ${{ github.event.inputs.os_installer_ubuntu_1804 }}"
+        installer_url="${{ github.event.inputs.base_url}}OpenStudio-${{ github.event.inputs.os_version}}${{ github.event.inputs.os_prerelease_tag}}%2B${{ github.event.inputs.os_build_sha}}-Ubuntu-18.04.deb"


maybe a bit fragile, if you forgot the trailing slash in base_url?

jmarrec · 2022-04-13T19:52:05Z

.github/workflows/manual_installer_all_platforms.yml

      run: |
        set -x
-        echo "Installer link: ${{ github.event.inputs.os_installer_ubuntu_1804 }}"
+        installer_url="${{ github.event.inputs.base_url}}OpenStudio-${{ github.event.inputs.os_version}}${{ github.event.inputs.os_prerelease_tag}}%2B${{ github.event.inputs.os_build_sha}}-Ubuntu-18.04.deb"


Same would need to be applied to all jobs

Suggested change

installer_url="${{ github.event.inputs.base_url}}OpenStudio-${{ github.event.inputs.os_version}}${{ github.event.inputs.os_prerelease_tag}}%2B${{ github.event.inputs.os_build_sha}}-Ubuntu-18.04.deb"

base_url="${{ github.event.inputs.base_url}}"

# remove trailing slash if any

base_url=${base_url%/}

installer_url="$base_url/OpenStudio-${{ github.event.inputs.os_version}}${{ github.event.inputs.os_prerelease_tag}}%2B${{ github.event.inputs.os_build_sha}}-Ubuntu-18.04.deb"

yeah, this would be good to add.

jmarrec · 2022-04-13T19:52:39Z

.github/workflows/manual_installer_all_platforms.yml

-      os_installer_ubuntu_1804:
-        description: 'Ubuntu 1804 (.deb) URL'
+      base_url:
+        description: 'The base url to use to download each platform'


Do we expect to use builds from s3 instead of a prerelease? just curious

yes, I recently used s3 as a test and works fine.

cf NatLabRockies/OpenStudio#4701 cf NatLabRockies/OpenStudio#4699

…urces into develop

tijcolem added 16 commits October 28, 2021 15:46

test using all supported platforms in GH actions

a9515e0

remove the branch name as not using it

e2d988a

Add different workflow title

49aaa6e

Merge pull request #1 from tijcolem/expand_platforms

3124cbd

Expand platforms

try and shorten the description input labels

cbbd260

fix yaml

2cb984f

remove extra 2004 block

dea721b

use curl vs wget on windows

78ce073

try again on windows download

817fa98

try again on windows download

77a127f

try and fix mac installer

4c7aae5

let's try and test using ruby bindings

e3bf56c

enable ruby binding test on all platforms

976c482

add both cli and ruby bindings tests

1975a70

fix mac nproc cmd

dcbbdd2

fix mac nproc again

8721379

tijcolem added the Other PR type: Something else, like maintenance of the repo, or just committing test results label Oct 29, 2021

tijcolem requested a review from jmarrec October 29, 2021 13:09

jmarrec requested changes Oct 29, 2021

View reviewed changes

tijcolem added 9 commits November 2, 2021 09:14

use single qt install script

9e6f748

add tags to cli and ruby and process these using tagged option

e7885eb

Merge remote-tracking branch 'nrel/develop' into develop

11600eb

make the all platform action use a single base url

61dff0d

add / to end of urls

8d3b3e0

fix description

a85e66c

add proper default value

c03a038

remove end /

8742b73

fix mac file type

e0f46c3

jmarrec reviewed Apr 13, 2022

View reviewed changes

Force an Always On Supply Air Fan Operating Mode Schedule for PTAC/PTHP

9a404be

cf NatLabRockies/OpenStudio#4701 cf NatLabRockies/OpenStudio#4699

tijcolem and others added 9 commits October 4, 2022 12:44

Merge remote-tracking branch 'remotes/nrel/develop' into develop

2d36564

updating ext for mac installer

626a501

update required to false for prerelease tag

287a76c

update 3.4.0 tests

9f02ada

Merge branch 'develop' of https://github.com/tijcolem/OpenStudio-reso…

1bfa708

…urces into develop

merge latest from nrel/OpenStudio-resources

4d7e35e

update to use 22.04 runner

4daa011

fixes for url paths and ruby action

cf87ef2

update mac runner to macos-11

2233455

-        installer_url="${{ github.event.inputs.base_url}}OpenStudio-${{ github.event.inputs.os_version}}${{ github.event.inputs.os_prerelease_tag}}%2B${{ github.event.inputs.os_build_sha}}-Ubuntu-18.04.deb"
+        base_url="${{ github.event.inputs.base_url}}"
+        # remove trailing slash if any
+        base_url=${base_url%/}
+        installer_url="$base_url/OpenStudio-${{ github.event.inputs.os_version}}${{ github.event.inputs.os_prerelease_tag}}%2B${{ github.event.inputs.os_build_sha}}-Ubuntu-18.04.deb"

SDK Installer test on all platforms #156

Are you sure you want to change the base?

SDK Installer test on all platforms #156

Uh oh!

Conversation

tijcolem commented Oct 29, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull request overview

Uh oh!

jmarrec left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tijcolem Nov 2, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tijcolem Apr 13, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jmarrec Apr 13, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

tijcolem commented Oct 29, 2021 •

edited

Loading

tijcolem Nov 2, 2021 •

edited

Loading

tijcolem Apr 13, 2022 •

edited

Loading

jmarrec Apr 13, 2022 •

edited

Loading