Conversation
Co-authored-by: abs2023 <alan@lumerin.io>
* edit readme * update to trigger build in gitlab 1 --------- Co-authored-by: abs2023 <alan@lumerin.io>
Co-authored-by: abs2023 <alan@lumerin.io>
* 1.1.0-STG Community Test (#6) * edit readme (#1) Co-authored-by: abs2023 <alan@lumerin.io> * Test sync (#3) * edit readme * update to trigger build in gitlab 1 --------- Co-authored-by: abs2023 <alan@lumerin.io> * update comment to unprotected branch (#5) --------- Co-authored-by: LumerinIO <102425763+LumerinIO@users.noreply.github.com> Co-authored-by: abs2023 <alan@lumerin.io> * Cicd environment (#10) * edit readme (#1) Co-authored-by: abs2023 <alan@lumerin.io> * Test sync (#3) * edit readme * update to trigger build in gitlab 1 --------- Co-authored-by: abs2023 <alan@lumerin.io> * update comment to unprotected branch (#5) * updated readme (#7) * update environemnt settings for CICD in gitlab --------- Co-authored-by: abs2023 <alan@lumerin.io> Co-authored-by: roguevader <157174704+roguevader@users.noreply.github.com> --------- Co-authored-by: roguevader <157174704+roguevader@users.noreply.github.com> Co-authored-by: LumerinIO <102425763+LumerinIO@users.noreply.github.com>
Co-authored-by: Oleksandr Shevchuk <lsheva7@gmail.com>
Co-authored-by: Oleksandr Shevchuk <lsheva7@gmail.com>
Initial Framework
| ); | ||
|
|
||
| const server = new Server(cache, loader, service, log.child({ module: "server" })); | ||
| log.info(`Starting app with config: ${JSON.stringify(config)}`); |
There was a problem hiding this comment.
Logging the entire configuration object can expose sensitive data in the logs, which is a security risk. Consider logging only non-sensitive parts of the configuration or masking sensitive values.
Recommended Change:
log.info(`Starting app with partial config: ${JSON.stringify({ ...config, sensitiveKey: '****' })}`);| log.info("Using custom multicall address", config.MULTICALL_ADDRESS); | ||
| chain.contracts = { | ||
| ...chain.contracts, | ||
| multicall3: { address: config.MULTICALL_ADDRESS as `0x${string}` }, | ||
| }; | ||
| log.info("Multicall address", config.MULTICALL_ADDRESS); |
There was a problem hiding this comment.
Directly using config.MULTICALL_ADDRESS without validation could lead to issues if the address is not properly formatted. This can cause the application to interact incorrectly with the blockchain.
Recommended Change:
Add validation for config.MULTICALL_ADDRESS to ensure it is a valid Ethereum address before using it in the application logic.
if (!isValidEthereumAddress(config.MULTICALL_ADDRESS)) {
throw new Error('Invalid MULTICALL_ADDRESS in configuration.');
}| import { type Static, Type } from "@sinclair/typebox"; | ||
| import envSchema from "env-schema"; | ||
|
|
||
| const schema = Type.Object({ | ||
| ADMIN_API_KEY: Type.String(), | ||
| CLONE_FACTORY_ADDRESS: Type.String(), | ||
| ETH_NODE_URL: Type.String(), | ||
| HASHRATE_ORACLE_ADDRESS: Type.String(), | ||
| FASTIFY_PLUGIN_TIMEOUT: Type.Integer({ default: 60000 }), | ||
| FASTIFY_CLOSE_GRACE_DELAY: Type.Integer({ default: 500 }), | ||
| LOG_LEVEL: Type.Union( | ||
| [ | ||
| Type.Literal("trace"), | ||
| Type.Literal("debug"), | ||
| Type.Literal("info"), | ||
| Type.Literal("warn"), | ||
| Type.Literal("error"), | ||
| Type.Literal("fatal"), | ||
| ], | ||
| { default: "info" } | ||
| ), | ||
| MULTICALL_ADDRESS: Type.Optional(Type.String()), | ||
| PORT: Type.Integer({ default: 3000 }), | ||
| }); | ||
|
|
||
| export type Config = Static<typeof schema>; | ||
|
|
||
| export const config = envSchema<Config>({ | ||
| schema, | ||
| dotenv: true, // load .env if it is there, default: false | ||
| }); |
There was a problem hiding this comment.
Security Enhancement for Sensitive Data
The handling of sensitive environment variables such as ADMIN_API_KEY and ETH_NODE_URL could be improved by implementing additional security measures. While the current schema validation ensures that the data conforms to the expected format, it does not encrypt or secure the data at rest or in transit.
Recommendation:
- Consider using environment-specific secrets management solutions (e.g., AWS Secrets Manager, HashiCorp Vault) to store and access sensitive keys and configurations.
- Ensure that sensitive data is not logged or exposed in error messages or logs.
| dotenv: true, // load .env if it is there, default: false | ||
| }); |
There was a problem hiding this comment.
Caution with .env File Loading in Production
The dotenv: true setting in envSchema configuration automatically loads environment variables from a .env file if present. This is convenient for development but can pose a security risk in production environments if not managed properly, as sensitive information might get exposed.
Recommendation:
- Ensure that
.envfiles are not used in production or are adequately secured. - Consider implementing checks to disable
.envfile loading in production environments to prevent accidental exposure of sensitive configuration.
| export const start = async (client: PublicClient, loader: ContractsLoader, indexer: Cache, log: FastifyBaseLogger) => { | ||
| log.info("Initial load of contracts"); | ||
|
|
||
| const res = await loader.loadAll(); | ||
| log.info("Loaded contracts", res.contracts.length); | ||
|
|
||
| for (const contract of res.contracts) { | ||
| updateContract(contract, Number(res.blockNumber), indexer, log); | ||
| } | ||
|
|
||
| await startWatchPromise(client, { | ||
| initialContractsToWatch: new Set(res.contracts.map((c) => c.id)), | ||
| onContractUpdate: async (contractAddr: string, blockNumber: number) => { | ||
| const contract = await loader.getContract(contractAddr as `0x${string}`); | ||
| updateContract(contract, blockNumber, indexer, log); | ||
| }, | ||
| onFeeUpdate: async (feeRateScaled: bigint) => { | ||
| const decimals = await loader.cloneFactory.read.VALIDATOR_FEE_DECIMALS(); | ||
| log.info("Fee rate updated", { feeRateScaled, decimals }); | ||
| indexer.setFeeRate({ value: feeRateScaled, decimals: BigInt(decimals) }); | ||
| }, | ||
| log, | ||
| }); |
There was a problem hiding this comment.
Lack of Error Handling
The function start does not implement error handling for asynchronous operations, which can lead to unhandled promise rejections and potential application crashes. This is particularly critical for operations like loader.loadAll() at line 11, which can fail due to external factors.
Recommendation: Wrap the asynchronous operations within start in try-catch blocks. Log the errors using log.error and consider appropriate error recovery or retry mechanisms. For example:
try {
const res = await loader.loadAll();
// further processing...
} catch (error) {
log.error('Failed to load contracts', error);
// handle error appropriately
}| export function mapFutureTerms(futureTerms: FutureTermsEntry | undefined): FutureTerms | undefined { | ||
| if (!futureTerms) { | ||
| return undefined; | ||
| } | ||
| const [, , speed, length, version, profitTarget] = futureTerms; | ||
| return { | ||
| speed: speed.toString(), | ||
| length: length.toString(), | ||
| version: version.toString(), | ||
| profitTarget: profitTarget.toString(), | ||
| }; |
There was a problem hiding this comment.
The function mapFutureTerms returns undefined if no futureTerms are provided. This approach requires that all calling functions are prepared to handle an undefined value, which might not always be the case and could lead to errors.
Recommended Solution:
Consider providing a default return value that represents an empty but valid FutureTerms object. This would prevent potential errors in parts of the application that may not handle undefined values well.
| private async getHashesPerTokenCached(): Promise<bigint> { | ||
| return await this.mutex.runExclusive(async () => { | ||
| if (this.priceExpirationTime > new Date() && this.pricePerTHInToken !== null) { | ||
| return this.pricePerTHInToken; | ||
| } | ||
| this.pricePerTHInToken = await this.getHashesPerToken(); | ||
| this.priceExpirationTime = new Date(Date.now() + this.ttl); | ||
| return this.pricePerTHInToken!; | ||
| }); |
There was a problem hiding this comment.
The use of await inside the mutex.runExclusive callback might be redundant. Since runExclusive itself returns a promise that resolves to the value returned by the passed async function, you can simplify the code by directly returning the result from the async function without an additional await. This can enhance readability and potentially improve performance by reducing the overhead of an unnecessary promise resolution.
Suggested Change:
private async getHashesPerTokenCached(): Promise<bigint> {
return this.mutex.runExclusive(async () => {
if (this.priceExpirationTime > new Date() && this.pricePerTHInToken !== null) {
return this.pricePerTHInToken;
}
this.pricePerTHInToken = await this.getHashesPerToken();
this.priceExpirationTime = new Date(Date.now() + this.ttl);
return this.pricePerTHInToken!;
});
}| if (this.priceExpirationTime > new Date() && this.pricePerTHInToken !== null) { | ||
| return this.pricePerTHInToken; | ||
| } | ||
| this.pricePerTHInToken = await this.getHashesPerToken(); | ||
| this.priceExpirationTime = new Date(Date.now() + this.ttl); |
There was a problem hiding this comment.
The method getHashesPerTokenCached uses new Date() for comparing the current time against priceExpirationTime and for setting the new expiration time. While this works correctly, using Date.now() for both getting the current timestamp and calculating the new expiration time can be more efficient and consistent. Date.now() returns the number of milliseconds since the Unix Epoch, which is suitable for direct arithmetic operations and comparisons.
Suggested Change:
if (this.priceExpirationTime > Date.now() && this.pricePerTHInToken !== null) {
return this.pricePerTHInToken;
}
this.pricePerTHInToken = await this.getHashesPerToken();
this.priceExpirationTime = Date.now() + this.ttl;
return this.pricePerTHInToken!;| price: string; | ||
| fee: string; | ||
| speed: string; // in hashes per second | ||
| length: string; // in seconds | ||
| profitTarget: string; // in percent |
There was a problem hiding this comment.
Issue: Use of String for Numerical Values
Fields such as price, fee, speed, length, and profitTarget are defined as string types, which are inherently numerical in nature. Using strings for these values can complicate arithmetic operations and comparisons, as they require explicit conversions to numbers, increasing the risk of runtime errors and reducing performance.
Recommendation: Consider using number or bigint for these fields if precision and large value handling are concerns. This change would enhance data handling and operations efficiency.
| purchaseTime: string; | ||
| endTime: string; | ||
| price: string; | ||
| fee: string; | ||
| speed: string; | ||
| length: string; |
There was a problem hiding this comment.
The fields purchaseTime, endTime, price, fee, speed, and length are all defined as string types, which might not be the most appropriate choice:
- Dates:
purchaseTimeandendTimeshould ideally be of aDatetype or a similar date-time structure to ensure consistency in handling dates. - Numeric values:
price,fee,speed, andlengthshould be typed asnumberto facilitate calculations and comparisons.
Recommended Change:
Refactor these fields to use more specific types (Date for date fields and number for numeric values) to enhance type safety and performance.
| ); | ||
|
|
||
| const server = new Server(cache, loader, service, log.child({ module: "server" })); | ||
| log.info(`Starting app with config: ${JSON.stringify(config)}`); |
There was a problem hiding this comment.
Logging configuration data directly with log.info can expose sensitive information in your logs, which is a security risk. Consider sanitizing or omitting sensitive data from logs to prevent potential information leakage.
Recommended Solution:
Create a utility function to sanitize configuration data by omitting or obfuscating sensitive values before logging. Use this function whenever you log configuration data.
| main().catch((err) => { | ||
| console.error("Exiting app due to error", err); | ||
| process.exit(1); | ||
| }); |
There was a problem hiding this comment.
The error handling in the main function abruptly exits the process which might not be suitable for a production environment as it can lead to an unexpected termination of the application without proper cleanup or shutdown procedures.
Recommended Solution:
Implement a more graceful shutdown process. This could involve logging the error, notifying administrators, and attempting to close resources or connections gracefully before exiting. Consider using process.on('uncaughtException') and process.on('unhandledRejection') for broader error handling across the application.
| const schema = Type.Object({ | ||
| ADMIN_API_KEY: Type.String(), | ||
| CLONE_FACTORY_ADDRESS: Type.String(), | ||
| ETH_NODE_URL: Type.String(), | ||
| HASHRATE_ORACLE_ADDRESS: Type.String(), | ||
| FASTIFY_PLUGIN_TIMEOUT: Type.Integer({ default: 60000 }), | ||
| FASTIFY_CLOSE_GRACE_DELAY: Type.Integer({ default: 500 }), | ||
| LOG_LEVEL: Type.Union( | ||
| [ | ||
| Type.Literal("trace"), | ||
| Type.Literal("debug"), | ||
| Type.Literal("info"), | ||
| Type.Literal("warn"), | ||
| Type.Literal("error"), | ||
| Type.Literal("fatal"), | ||
| ], | ||
| { default: "info" } | ||
| ), | ||
| MULTICALL_ADDRESS: Type.Optional(Type.String()), | ||
| PORT: Type.Integer({ default: 3000 }), | ||
| }); |
There was a problem hiding this comment.
Suggestion for Enhanced Validation of Critical Environment Variables
The schema for environment variables lacks additional validation for critical keys such as ADMIN_API_KEY and ETH_NODE_URL. It's important to ensure these values meet specific criteria (e.g., length, format) to avoid configuration errors or security issues.
Recommended Solution:
Implement regex patterns or custom validation functions for critical environment variables to ensure they conform to expected formats and standards. For example:
ADMIN_API_KEY: Type.String({ minLength: 32, pattern: /^[a-zA-Z0-9]{32,}$/ }),
ETH_NODE_URL: Type.String({ format: 'uri' })| dotenv: true, // load .env if it is there, default: false | ||
| }); |
There was a problem hiding this comment.
Caution with .env File Loading in Production
The dotenv: true option in envSchema function loads environment variables from a .env file if present. While this is convenient for development, it can pose risks in production by unintentionally overriding environment variables.
Recommended Solution:
Consider controlling the loading of .env files based on the environment. For instance, only enable .env loading in development mode:
dotenv: process.env.NODE_ENV === 'development'| export const start = async (client: PublicClient, loader: ContractsLoader, indexer: Cache, log: FastifyBaseLogger) => { | ||
| log.info("Initial load of contracts"); | ||
|
|
||
| const res = await loader.loadAll(); | ||
| log.info("Loaded contracts", res.contracts.length); | ||
|
|
||
| for (const contract of res.contracts) { | ||
| updateContract(contract, Number(res.blockNumber), indexer, log); | ||
| } | ||
|
|
||
| await startWatchPromise(client, { | ||
| initialContractsToWatch: new Set(res.contracts.map((c) => c.id)), | ||
| onContractUpdate: async (contractAddr: string, blockNumber: number) => { | ||
| const contract = await loader.getContract(contractAddr as `0x${string}`); | ||
| updateContract(contract, blockNumber, indexer, log); | ||
| }, | ||
| onFeeUpdate: async (feeRateScaled: bigint) => { | ||
| const decimals = await loader.cloneFactory.read.VALIDATOR_FEE_DECIMALS(); | ||
| log.info("Fee rate updated", { feeRateScaled, decimals }); | ||
| indexer.setFeeRate({ value: feeRateScaled, decimals: BigInt(decimals) }); | ||
| }, | ||
| log, | ||
| }); |
There was a problem hiding this comment.
Error Handling Improvement
The start function lacks comprehensive error handling for asynchronous operations. When interacting with external services such as loader.loadAll() and startWatchPromise, errors can occur that are not caught, potentially causing the application to crash or enter an inconsistent state.
Recommendation:
Wrap the asynchronous calls within try-catch blocks to handle possible exceptions. Log the errors using log.error and consider appropriate recovery or retry mechanisms depending on the nature of the error.
Example:
try {
const res = await loader.loadAll();
// process res
} catch (error) {
log.error('Failed to load contracts', error);
// handle error appropriately
}| if (this.priceExpirationTime > new Date() && this.pricePerTHInToken !== null) { | ||
| return this.pricePerTHInToken; | ||
| } | ||
| this.pricePerTHInToken = await this.getHashesPerToken(); | ||
| this.priceExpirationTime = new Date(Date.now() + this.ttl); | ||
| return this.pricePerTHInToken!; |
There was a problem hiding this comment.
The caching mechanism in the getHashesPerTokenCached method uses a mutex to prevent data races, which is good. However, the logic to check if the price is expired and to fetch a new price if necessary could be optimized. The current implementation always updates priceExpirationTime after fetching a new price, which might lead to unnecessary updates if the fetched price hasn't changed.
Recommended Solution:
Consider adding a condition to check if the fetched price differs from the current pricePerTHInToken before updating priceExpirationTime. This would prevent unnecessary updates and ensure that the cache expiration is only reset when new data is actually fetched.
| price: string; | ||
| fee: string; | ||
| speed: string; // in hashes per second | ||
| length: string; // in seconds | ||
| profitTarget: string; // in percent |
There was a problem hiding this comment.
Fields such as price, fee, speed, length, and profitTarget are defined as strings, which might not be the most appropriate type considering these represent numeric values. Using strings for numeric data can lead to unnecessary type conversions and can complicate arithmetic operations and comparisons.
Recommended Solution:
Consider changing these fields to number type to ensure type safety and simplify operations involving these values.
| validator: `0x${string}`; | ||
| }; | ||
|
|
||
| /** Represents an internal stats object.**/ | ||
| export type Stats = { | ||
| successCount: string; | ||
| failCount: string; | ||
| }; | ||
|
|
||
| /** Represents an internal contract history object.**/ | ||
| export type ContractHistory = { | ||
| buyer: string; | ||
| validator: string; |
There was a problem hiding this comment.
The validator field in HashrateContract is defined as a template literal type 0x${string}, suggesting a specific format (likely a hexadecimal string). However, in ContractHistory, the validator field is simply a string. This inconsistency can lead to confusion and potential errors when handling validator data across different parts of the application.
Recommended Solution:
Ensure consistency in the type definition of validator across different types. If a specific format is required (e.g., hexadecimal), enforce this format consistently or use a more descriptive type or validation method to ensure data integrity.
| purchaseTime: string; | ||
| endTime: string; |
There was a problem hiding this comment.
The fields purchaseTime and endTime are currently typed as string, which might lead to inconsistencies in date handling. Consider using Date or a string format compliant with ISO 8601 to ensure robust date operations and comparisons.
Recommended Change:
purchaseTime: Date;
endTime: Date;| price: string; | ||
| fee: string; | ||
| speed: string; | ||
| length: string; |
There was a problem hiding this comment.
The fields price, fee, speed, and length are typed as string, which could lead to unnecessary type conversions and potential errors in calculations, especially for financial data. Consider using number for general numeric values or bigint for large numbers to ensure precision and ease of calculations.
Recommended Change:
price: number;
fee: number;
speed: number;
length: number;
No description provided.