Issue 6284: Add tables without using templates #6360
Conversation
rtjd6554
added
the
needs-reviewer
ca61688
removed
the
needs-reviewer
| InstanceProperties instanceProperties = S3InstanceProperties.loadGivenInstanceId(s3Client, instanceId); | ||
| TableProperties tableProperties = new TableProperties(instanceProperties, loadProperties(tablePropertiesFile)); | ||
| TableProperties tableProperties = LoadLocalProperties.loadTableFromPropertiesFileNoValidation(instanceProperties, tablePropertiesFile).properties(); | ||
| tableProperties.setSchema(Schema.load(schemaFile)); |
There was a problem hiding this comment.
This is overriding the schema read from the local folder.
| sed \ | ||
| -e "s|^sleeper.table.name=.*|sleeper.table.name=${TABLE_NAME}|" \ | ||
| "${TEMPLATE_DIR}/tableproperties.template" \ | ||
| > "${TABLE_PROPERTIES}" |
There was a problem hiding this comment.
This is forcing you to use the "generated" folder, which is internal to Sleeper. There could be files there already, which it shouldn't be necessary to overwrite. If you put your files in there, they would be overwritten by other scripts. You should be able to pass in whatever files you want, wherever you've put them.
| sed \ | ||
| -e "s|^sleeper.table.name=.*|sleeper.table.name=${TABLE_NAME}|" \ | ||
| "${TEMPLATE_DIR}/tableproperties.template" \ | ||
| > "${TABLE_PROPERTIES}" |
There was a problem hiding this comment.
The documentation needs updating based on the changes to this script.
| sed \ | ||
| -e "s|^sleeper.table.name=.*|sleeper.table.name=${TABLE_NAME}|" \ | ||
| "${TEMPLATE_DIR}/tableproperties.template" \ | ||
| > "${TABLE_PROPERTIES}" |
There was a problem hiding this comment.
The issue suggested we could keep the existing behaviour of using the template files as the defaults. This has deleted that behaviour, so it's no longer possible to add a table without manually creating the configuration files.
Make sure you have checked all steps below.
Issue
Feature". Note that before an issue is finished, you can still make a pull request by raising a separate issue
for your progress.
Tests
Documentation
separate issue for that below.