Skip to main content

balm_aggr_consol

1. Description​

Balm_aggr_consol program reads the aggregator output and it generates a consolidated summary of it. It can take both maturity or non maturity aggregated output as input. The output generated follows the similar structure as the input. It converts the currency type for the given input files and write the converted values consolidated summary to the respective output files.

2. Program location​

<installation_path>/balm_aggr_consol

3. Input​

Click ⬇️ to download the sample data.

3.1 Mandatory Parameters​

#ParametersDescriptionExample
1as_on_dateThe date for which the program has to run.04-10-2023
2input_file_pathPath to the input files.test-bed/aggregated/aggregated
3output_file_pathPath to the output file.test-bed/output/aggregated
4log_file_pathPath to write logs.test-bed/output/log.txt
5diagnostics_log_filePath to write diagnostics logs.test-bed/output/diag_log.txt
6exchange_rate_filePath to the exchange rate file. 🔗test-bed/1000ExchangeRate.txt
7consol_config_pathPath to the consol-config file.🔗test-bed/consol-config.txt
8is_maturityIf is_maturity_flag is set, then the input file is of maturity type else it is of non maturity type.true

3.2 Non Mandatory Parameters​

#ParametersDescriptionDefault valueExample
1input_files_typeIndicates the file type: BALM or BRWISEBALMBALM or BRWISE
2log_levelLevel of diagnostics written to the log file.infoerror/warn/info/debug/trace/none
3is_perf_diagnostics_enabledThe flag that decides whether performance diagnostics will be written to the diagnostics log file.falsetrue or false
4interest_decimal_precisionThe value which decides the interest rate round off precision.44
5amount_decimal_precisionThe value which decides the amount value round off precision.22

Consol config file:​

Consol config file contains three columns:

#Field NameDescriptionExample
1source_currencyThe currency type from the input is matched with this field. It acts as a source type for the currency conversion.USD
2consol_ccyThis field act as destination type for the currency conversion.INR
3display_consol_ccyAfter converting the currency from source type to destination type using the exchange rate, the converted currency type is displayed as display_consol_ccy.RUP

Example: if the 'curr|amt' in input file is 'USD|10', then the USD is searched as key in the config file. Then for all the matches, it gets converted to respective consol_ccy and displayed as display_consol_ccy. For the above example, 10 USD is converted to 800 INR using exchange rate(USD|INR|80.0) from exchange_rate_file. The final output is: 'RUP|800'.

Note: If source_key is matched multiple times, then for all the different consol_ccy destination type, separate conversion takes place.

4. Output​

The number of output files generated and each file's number of columns & order depends on the type of input. The output generated follows the similar structure as the input. It just changes the currency type as per the consol-config file and consolidates the output depending on the key value.

Note: The output files name are slightly different here in comparision with balm aggregator output files name. Here, it will contain '-consol' as postfix. Example: op-1-consol.txt and op-summary-consol.txt.

The two input parameters: is_maturity(true or false) and input_files_type(BALM or BRWISE), is used to define the type of input. Four possible scenarios can be there:

4.1 is_maturity=true and input_files_type:BALM​

In this case, the aggregated output is from the balm_aggr_m program.🔗 In this case, the input is 12 master files and 1 aggregated-summary file. Therefore, 12 master consolidated files and 1 aggreagated-summary consolidated file is generated. The 12 master consolidated files will get uploaded to respective MasterInflowConvertedN and MasterOutflowConvertedN table. N representing the file number. Aggreagated-summary consolidated file will get uploaded to tblProductTotalsConverted table.

The 12 master consolidated file follows the following order:
llg.category|as_on_date|llg.currency|Mater|cashflow_aggregated_type|amt|rate|date|amt|rate|date|...

The 1 aggreagated-summary consolidated file follows the following order:
llg.category|as_on_date|llg.currency|cashflow_aggregated_type|Mater|cashflow_type|amount|rate

4.2 is_maturity=true and input_files_type:BRWISE​

In this case, the aggregated output is from the balm_aggr_dim_m program.🔗 In this case, the input is 12 master files and 1 aggregated-summary file. Therefore, 12 master consolidated files and 1 aggreagated-summary consolidated file is generated. The 12 master consolidated files will get uploaded to respective DimMasterInflowConvertedN and DimMasterOutflowConvertedN table. N representing the file number. Aggreagated-summary consolidated file will get uploaded to DimMasterProductTotalsConverted table.

The 12 master consolidated file follows the following order:
llg.category|llg.item_id|llg.dim_id|as_on_date|llg.currency|cashflow_aggregated_type|amt|rate|date|amt|rate|date|...

The 1 aggreagated-summary consolidated file follows the following order:
llg.category|llg.item_id|llg.dim_id|as_on_date|llg.currency|cashflow_aggregated_type|cashflow_type|amount|rate

Description

4.3 is_maturity=false and input_files_type:BALM​

In this case, the aggregated output is from the balm_aggr_nm program.🔗 In this case, the input is only 1 aggregated-summary file. Therefore, only 1 aggreagated-summary consolidated file is generated. Aggreagated-summary consolidated file will get uploaded to tblProductTotalsConverted table.

The 1 aggreagated-summary consolidated file follows the following order:
llg.category|as_on_date|llg.currency|ALL|MASTER|llg.cf_type|amount|rate

4.4 is_maturity=false and input_files_type:BRWISE​

In this case, the aggregated output is from the balm_aggr_dim_nm program.🔗 In this case, the input is only 1 aggregated-summary file. Therefore, only 1 aggreagated-summary consolidated file is generated. Aggreagated-summary consolidated file will get uploaded to DimMasterProductTotalsConverted table.

The 1 aggreagated-summary consolidated file follows the following order:
llg.category|llg.item_id|llg.dim_id|as_on_date|llg.currency|ALL|llg.cf_type|amount|rate

Output derivation logic:​

For a given input row in a particular file, read the currency type. Search for this currency type in the consol config file 'source_currency' column. Get all the matching consol_ccy destination type. Now for every consol_ccy destination, make use of the exchange rate file to get the exchange rate. Use this exchange rate to convert the subsequent amount present in the given input row. After converting the value, write it in similar structure to the respective output file row.

For example,
8888|28-02-2023|USD|Master|INT|1000.00|2000.0|...

Search for USD in the config file. Lets say the config file looks like this:
USD|INR|RUP
USD|EUR|EUR
We get two consol_ccy destination type INR and EUR. 

Lets say the exchange file looks like this:
USD|INR|83.26
USD|EUR|0.95 
So we convert subsequent amount and stamp them with display type:
8888|28-02-2023|RUP|Master|INT|83187.65 (1000.00*83.26)|166371.90 (2000.00*83.26)|...
8888|28-02-2023|EUR|Master|INT|947.08 (1000.00*0.95)|1894.17 (2000.00*0.95)|...

Note: If after conversion, the aggegation key(For eg: llg.category,as_on_date,llg.currency,..) becomes same as some other aggregation key, then we aggregate those amounts together.

For example,
8888|28-02-2023|USD|Master|INT|1000.00|2000.0|...
8888|28-02-2023|EUR|Master|INT|1000.00|2000.0|...
Search for USD in the config file. Lets say the config file looks like this:
USD|INR|RUP
EUR|INR|RUP
We get consol_ccy destination type INR as destination for first row. 

Lets say the exchange file looks like this:
USD|INR|83.26
EUR|INR|87.38
So we convert subsequent amount and stamp them with display type:
8888|28-02-2023|RUP|Master|INT|83187.65 (1000.00*83.26)|166371.90 (2000.00*83.26)|...
Then when we go the 2nd input row, we get the consol_ccy destination type INR again. So we convert subsequent amount and stamp them with display type:
8888|28-02-2023|RUP|Master|INT|87903.64 (1000.00*87.38)|175807.28 (2000.00*87.38)|...
But this time the aggregation key matches with the first row aggregation key (8888|28-02-2023|RUP|Master|INT|), therefore we aggregate them together and write them as one:
8888|28-02-2023|RUP|Master|INT|171091.29 (83187.65+87903.64)|342179.18 (166371.90+175807.28)|...

Note: Also, if we don't get any match for the currency type in the consol config file, then those row data will be lost.

For example,
8888|28-02-2023|USD|Master|INT|1000.00|2000.0|...
Search for USD in the config file. Lets say the config file looks like this:
EUR|INR|RUP
EUR|USD|USD
We get no matches for the USD. So, for that particular column, we write nothing to the respective output file.

NOTE: No health check report is getting generated for this aggregator.

5. Process configuration in streamdef table:​

  • is_maturity=false and input_files_type:BALM

      .........
      "processName": "GL Consol aggr",
      "processId": "3",
      "processBinary": "{SH_PROGRAMS}/balm_aggr_consol",
      "processArguments": [
          "--as-on-date",
          "{AsOnDate}",
          "--input-file",
          "{SH_SUMMARYDATA}/IND/BALM/{ddmmyyyy}/balm_gl",
          "--output-file",
          "{SH_SUMMARYDATA}/IND/BALM/{ddmmyyyy}/conv_balm_gl",
          "--log-file",
          "{SH_LOGS}/IND/BALM/{ddmmyyyy}/conv_balm_gl_log.txt",
          "--diagnostics-log-file",
          "{SH_LOGS}/IND/BALM/{ddmmyyyy}/conv_balm_gl_diag_log.txt",
          "--exchange-rate-file",
          "{SH_RESOURCES}/IND/{ddmmyyyy}/exchangeRate.txt",
          "--consol-config-file",
          "{SH_RESOURCES}/IND/OPERATIONAL-FILES/MASTER-FILES/consol_config.txt",
          "--is-maturity",
          "false"
      ],
      "processDependencies": [],
      "processReport": "",
      "processType": "AGGR",
      "infoLog": "{SH_LOGS}/IND/BALM/{ddmmyyyy}/conv_balm_gl_log.txt",
      "errorLog": ""
      .........

  • is_maturity=false and input_files_type:BRWISE

      .........
      "processName": "GL Dimensional Consol aggr",
      "processId": "4",
      "processBinary": "{SH_PROGRAMS}/balm_aggr_consol",
      "processArguments": [
          "--as-on-date",
          "{AsOnDate}",
          "--input-file",
          "{SH_SUMMARYDATA}/IND/BALM/{ddmmyyyy}/dim_balm_gl",
          "--output-file",
          "{SH_SUMMARYDATA}/IND/BALM/{ddmmyyyy}/conv_dim_balm_gl",
          "--log-file",
          "{SH_LOGS}/IND/BALM/{ddmmyyyy}/conv_dim_balm_gl_log.txt",
          "--diagnostics-log-file",
          "{SH_LOGS}/IND/BALM/{ddmmyyyy}/conv_dim_balm_gl_diag_log.txt",
          "--exchange-rate-file",
          "{SH_RESOURCES}/IND/{ddmmyyyy}/exchangeRate.txt",
          "--consol-config-file",
          "{SH_RESOURCES}/IND/OPERATIONAL-FILES/MASTER-FILES/consol_config.txt",
          "--is-maturity",
          "false",
          "--input-files-type",
          "BRWISE"
      ],
      "processDependencies": [],
      "processReport": "",
      "processType": "AGGR",
      "infoLog": "{SH_LOGS}/IND/BALM/{ddmmyyyy}/conv_dim_balm_gl_log.txt",
      "errorLog": ""
      .........

  • is_maturity=true and input_files_type:BALM

      .........
      "processName": "TD Consol aggr",
      "processId": "3",
      "processBinary": "{SH_PROGRAMS}/balm_aggr_consol",
      "processArguments": [
          "--as-on-date",
          "{AsOnDate}",
          "--input-file",
          "{SH_SUMMARYDATA}/IND/BALM/{ddmmyyyy}/balm_td",
          "--output-file",
          "{SH_SUMMARYDATA}/IND/BALM/{ddmmyyyy}/conv_balm_td",
          "--log-file",
          "{SH_LOGS}/IND/BALM/{ddmmyyyy}/conv_balm_td_log.txt",
          "--diagnostics-log-file",
          "{SH_LOGS}/IND/BALM/{ddmmyyyy}/conv_balm_td_diag_log.txt",
          "--exchange-rate-file",
          "{SH_RESOURCES}/IND/{ddmmyyyy}/exchangeRate.txt",
          "--consol-config-file",
          "{SH_RESOURCES}/IND/OPERATIONAL-FILES/MASTER-FILES/consol_config.txt",
          "--is-maturity",
          "true"
      ],
      "processDependencies": [],
      "processReport": "",
      "processType": "AGGR",
      "infoLog": "{SH_LOGS}/IND/BALM/{ddmmyyyy}/conv_balm_td_log.txt",
      "errorLog": ""
      .........

  • is_maturity=true and input_files_type:BRWISE

      .........
      "processName": "TD Dimensional Consol aggr",
      "processId": "4",
      "processBinary": "{SH_PROGRAMS}/balm_aggr_consol",
      "processArguments": [
          "--as-on-date",
          "{AsOnDate}",
          "--input-file",
          "{SH_SUMMARYDATA}/IND/BALM/{ddmmyyyy}/dim_balm_td",
          "--output-file",
          "{SH_SUMMARYDATA}/IND/BALM/{ddmmyyyy}/conv_dim_balm_td",
          "--log-file",
          "{SH_LOGS}/IND/BALM/{ddmmyyyy}/conv_dim_balm_td_log.txt",
          "--diagnostics-log-file",
          "{SH_LOGS}/IND/BALM/{ddmmyyyy}/conv_dim_balm_td_diag_log.txt",
          "--exchange-rate-file",
          "{SH_RESOURCES}/IND/{ddmmyyyy}/exchangeRate.txt",
          "--consol-config-file",
          "{SH_RESOURCES}/IND/OPERATIONAL-FILES/MASTER-FILES/consol_config.txt",
          "--is-maturity",
          "true",
          "--input-files-type",
          "BRWISE"
      ],
      "processDependencies": [],
      "processReport": "",
      "processType": "AGGR",
      "infoLog": "{SH_LOGS}/IND/BALM/{ddmmyyyy}/conv_dim_balm_td_log.txt",
      "errorLog": ""
      .........