[2021/04/01-23:12:15.372] [FT-TrainWorkThread-ZYU2lQqF-67794] [INFO] [dku.analysis.prediction]  - ******************************************
[2021/04/01-23:12:15.372] [FT-TrainWorkThread-ZYU2lQqF-67794] [INFO] [dku.analysis.prediction]  - ** Start train session s2
[2021/04/01-23:12:15.372] [FT-TrainWorkThread-ZYU2lQqF-67794] [INFO] [dku.analysis.prediction]  - ******************************************
[2021/04/01-23:12:15.373] [FT-TrainWorkThread-ZYU2lQqF-67794] [INFO] [dku.analysis.splits] T-0IfwLWGQ - [ct: 2] Search for split: p=type=SPLIT_SINGLE_DATASET,split=RANDOM,splitBeforePrepare=true,ds=flight_joined_prepared,sel=(method=head-s,records=100000),r=0.8,s=1337 i=3c954e05fb55736ff7ecff41a6c5c14a-0
[2021/04/01-23:12:15.374] [FT-TrainWorkThread-ZYU2lQqF-67794] [INFO] [dku.analysis.splits] T-0IfwLWGQ - [ct: 3] Checking if splits are up to date. Policy: type=SPLIT_SINGLE_DATASET,split=RANDOM,splitBeforePrepare=true,ds=flight_joined_prepared,sel=(method=head-s,records=100000),r=0.8,s=1337, instance id: 3c954e05fb55736ff7ecff41a6c5c14a-0
[2021/04/01-23:12:15.374] [FT-TrainWorkThread-ZYU2lQqF-67794] [INFO] [dku.analysis.splits] T-0IfwLWGQ - [ct: 3] Search for split: p=type=SPLIT_SINGLE_DATASET,split=RANDOM,splitBeforePrepare=true,ds=flight_joined_prepared,sel=(method=head-s,records=100000),r=0.8,s=1337 i=3c954e05fb55736ff7ecff41a6c5c14a-0
[2021/04/01-23:12:15.374] [FT-TrainWorkThread-ZYU2lQqF-67794] [INFO] [dku.analysis.splits] T-0IfwLWGQ - [ct: 3] Search for split: p=type=SPLIT_SINGLE_DATASET,split=RANDOM,splitBeforePrepare=true,ds=flight_joined_prepared,sel=(method=head-s,records=100000),r=0.8,s=1337 i=3c954e05fb55736ff7ecff41a6c5c14a-0
[2021/04/01-23:12:15.375] [FT-TrainWorkThread-ZYU2lQqF-67794] [INFO] [dku.analysis.splits] T-0IfwLWGQ - [ct: 4] Checking if splits are up to date. Policy: type=SPLIT_SINGLE_DATASET,split=RANDOM,splitBeforePrepare=true,ds=flight_joined_prepared,sel=(method=head-s,records=100000),r=0.8,s=1337, instance id: 3c954e05fb55736ff7ecff41a6c5c14a-0
[2021/04/01-23:12:15.375] [FT-TrainWorkThread-ZYU2lQqF-67794] [INFO] [dku.analysis.splits] T-0IfwLWGQ - [ct: 4] Search for split: p=type=SPLIT_SINGLE_DATASET,split=RANDOM,splitBeforePrepare=true,ds=flight_joined_prepared,sel=(method=head-s,records=100000),r=0.8,s=1337 i=3c954e05fb55736ff7ecff41a6c5c14a-0
[2021/04/01-23:12:15.376] [FT-TrainWorkThread-ZYU2lQqF-67794] [INFO] [dku.analysis.splits] T-0IfwLWGQ - [ct: 5] Search for split: p=type=SPLIT_SINGLE_DATASET,split=RANDOM,splitBeforePrepare=true,ds=flight_joined_prepared,sel=(method=head-s,records=100000),r=0.8,s=1337 i=3c954e05fb55736ff7ecff41a6c5c14a-0
[2021/04/01-23:12:15.377] [FT-TrainWorkThread-ZYU2lQqF-67794] [INFO] [dku.analysis.ml.python] T-0IfwLWGQ - Joining processing thread ...
[2021/04/01-23:12:15.378] [MRT-67796] [INFO] [dku.analysis.ml.python]  - Running a preprocessing set: pp2 in /data/home/dkuacademy/Library/dss-9.0/design/analysis-data/QSDE_SF_TO_FS_V2/oSylVxXB/0IfwLWGQ/sessions/s2/pp2
[2021/04/01-23:12:15.378] [MRT-67796] [INFO] [dku.block.link]  - Started a socket on port 36279
[2021/04/01-23:12:15.379] [MRT-67796] [INFO] [dku.ml.kernel]  - Writing output of python-single-command-kernel to /data/home/dkuacademy/Library/dss-9.0/design/analysis-data/QSDE_SF_TO_FS_V2/oSylVxXB/0IfwLWGQ/sessions/s2/pp2/train.log
[2021/04/01-23:12:15.379] [MRT-67796] [INFO] [dku.code.envs.resolution]  - Executing Python activity in builtin env
[2021/04/01-23:12:15.379] [MRT-67796] [WARN] [dku.code.projectLibs]  - External libraries file not found: /data/home/dkuacademy/Library/dss-9.0/design/config/projects/QSDE_SF_TO_FS_V2/lib/external-libraries.json
[2021/04/01-23:12:15.379] [MRT-67796] [INFO] [dku.code.projectLibs]  - EXTERNAL LIBS FROM QSDE_SF_TO_FS_V2 is {"gitReferences":{},"pythonPath":["python"],"rsrcPath":["R"],"importLibrariesFromProjects":[]}
[2021/04/01-23:12:15.380] [MRT-67796] [INFO] [dku.code.projectLibs]  - chunkFolder is /data/home/dkuacademy/Library/dss-9.0/design/config/projects/QSDE_SF_TO_FS_V2/lib/R
[2021/04/01-23:12:15.380] [MRT-67796] [INFO] [dku.python.single_command.kernel]  - Starting Python process for kernel  python-single-command-kernel
[2021/04/01-23:12:15.380] [MRT-67796] [INFO] [dip.tickets]  - Creating API ticket for analysis-ml-QSDE_SF_TO_FS_V2-f1l8K58 on behalf of tara id=analysis-ml-QSDE_SF_TO_FS_V2-f1l8K58_vIPX9bGwXC45
[2021/04/01-23:12:15.380] [MRT-67796] [INFO] [dku.security.process]  - Starting process (regular)
[2021/04/01-23:12:15.382] [MRT-67796] [INFO] [dku.security.process]  - Process started with pid=5376
[2021/04/01-23:12:15.382] [MRT-67796] [INFO] [dku.processes.cgroups]  - Will use cgroups []
[2021/04/01-23:12:15.382] [MRT-67796] [INFO] [dku.processes.cgroups]  - Applying rules to used cgroups: []
[2021/04/01-23:12:15.384] [KNL-python-single-command-kernel-monitor-67803] [INFO] [dku.resourceusage]  - Reporting start of CRU:{"context":{"type":"ANALYSIS_ML_TRAIN","authIdentifier":"tara","projectKey":"QSDE_SF_TO_FS_V2","analysisId":"oSylVxXB","mlTaskId":"0IfwLWGQ","sessionId":"s2"},"type":"LOCAL_PROCESS","id":"DPkaMTEDmvR8Jj5M","startTime":1617311535384,"localProcess":{"cpuCurrent":0.0}}
[2021/04/01-23:12:15.387] [process-resource-monitor-5376-67805] [DEBUG] [dku.resource]  - Process stats for pid 5376: {"pid":5376,"commandName":"/data/home/dkuacademy/Library/dss-9.0/design/bin/python","cpuUserTimeMS":0,"cpuSystemTimeMS":0,"cpuChildrenUserTimeMS":0,"cpuChildrenSystemTimeMS":0,"cpuTotalMS":0,"cpuCurrent":0.0,"vmSizeMB":18,"vmRSSMB":3,"vmHWMMB":3,"vmRSSAnonMB":1,"vmDataMB":1,"vmSizePeakMB":18,"vmRSSPeakMB":3,"vmRSSTotalMBS":0,"majorFaults":0,"childrenMajorFaults":0}
Installing debugging signal handler
[2021-04-01 23:12:15,875] [5376/MainThread] [INFO] [dataiku.base.socket_block_link] Connecting to localhost (127.0.0.1) at port 36279
[2021-04-01 23:12:15,875] [5376/MainThread] [INFO] [dataiku.base.socket_block_link] Connected to localhost (127.0.0.1) at port 36279
[2021/04/01-23:12:15.876] [MRT-67796] [INFO] [dku.link.secret_protected]  - Connected to kernel
[2021/04/01-23:12:15.878] [MRT-67796] [INFO] [dku.block.link.interaction]  - Execute link command respClazz=true respTypeToken=false respIsString=false is=false asyncInputStream=false os=false
[2021-04-01 23:12:16,122] [5376/MainThread] [INFO] [dataiku.doctor.utils.dku_pickle] Setting cloudpickle as the pickling tool
[2021-04-01 23:12:16,138] [5376/MainThread] [INFO] [root] Running analysis command: train_prediction_models_nosave
[2021-04-01 23:12:16,138] [5376/MainThread] [INFO] [dataiku.doctor.diagnostics.diagnostics] enabling diagnostic callback: DatasetSanityCheckDiagnostic of type DiagnosticType.ML_DIAGNOSTICS_DATASET_SANITY_CHECKS
[2021-04-01 23:12:16,138] [5376/MainThread] [INFO] [dataiku.doctor.diagnostics.diagnostics] enabling diagnostic callback: ClassifierAccuracyCheckDiagnostic of type DiagnosticType.ML_DIAGNOSTICS_MODEL_CHECK
[2021-04-01 23:12:16,138] [5376/MainThread] [INFO] [dataiku.doctor.diagnostics.diagnostics] enabling diagnostic callback: RegressionR2CheckDiagnostic of type DiagnosticType.ML_DIAGNOSTICS_MODEL_CHECK
[2021-04-01 23:12:16,138] [5376/MainThread] [INFO] [dataiku.doctor.diagnostics.diagnostics] enabling diagnostic callback: LeakageDiagnostic of type DiagnosticType.ML_DIAGNOSTICS_LEAKAGE_DETECTION
[2021-04-01 23:12:16,138] [5376/MainThread] [INFO] [dataiku.doctor.diagnostics.diagnostics] enabling diagnostic callback: TreeOverfitDiagnostic of type DiagnosticType.ML_DIAGNOSTICS_TRAINING_OVERFIT
[2021-04-01 23:12:16,138] [5376/MainThread] [INFO] [dataiku.doctor.diagnostics.diagnostics] enabling diagnostic callback: MLAssertionsDiagnostic of type DiagnosticType.ML_DIAGNOSTICS_ML_ASSERTIONS
[2021-04-01 23:12:16,139] [5376/MainThread] [INFO] [dataiku.doctor.commands] PPS is {u'preprocessingFitSampleSeed': 1337, u'feature_selection_params': {u'custom_params': {u'code': u'# type your code here'}, u'pca_params': {u'variance_proportion': 0.9, u'n_features': 25}, u'random_forest_params': {u'depth': 10, u'n_features': 25, u'n_trees': 30}, u'lasso_params': {u'alpha': [0.01, 0.1, 1.0, 10.0, 100.0], u'cross_validate': True}, u'method': u'NONE', u'correlation_params': {u'max_abs_correlation': 1.0, u'n_features': 25, u'min_abs_correlation': 0.0}}, u'preprocessingFitSampleRatio': 1.0, u'reduce': {u'enabled': False, u'kept_variance': 0.0}, u'skipPreprocessing': False, u'target_remapping': [], u'per_feature': {u'Origin': {u'missing_impute_with': u'MODE', u'autoReason': u'REJECT_CARDINALITY', u'hash_whole_categories': True, u'nb_bins_hashing': 1048576, u'customHandlingCode': u'', u'category_handling': u'DUMMIFY', u'cumulative_proportion': 0.95, u'sendToInput': u'main', u'state': {u'userModified': False, u'recordedMeaning': u'Text', u'autoModifiedByDSS': False}, u'missing_handling': u'NONE', u'max_nb_categories': 100, u'customProcessorWantsMatrix': False, u'dummy_clip': u'MAX_NB_CATEGORIES', u'role': u'REJECT', u'max_cat_safety': 200, u'min_samples': 10, u'type': u'CATEGORY', u'dummy_drop': u'NONE'}, u'dep_hour': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'INPUT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'LongMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'UniqueCarrier': {u'missing_impute_with': u'MODE', u'customProcessorWantsMatrix': False, u'hash_whole_categories': True, u'nb_bins_hashing': 1048576, u'customHandlingCode': u'', u'category_handling': u'DUMMIFY', u'cumulative_proportion': 0.95, u'sendToInput': u'main', u'state': {u'userModified': False, u'recordedMeaning': u'Text', u'autoModifiedByDSS': False}, u'missing_handling': u'NONE', u'max_nb_categories': 5, u'dummy_clip': u'MAX_NB_CATEGORIES', u'role': u'INPUT', u'max_cat_safety': 200, u'min_samples': 10, u'type': u'CATEGORY', u'dummy_drop': u'DROP'}, u'FlightNum': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'INPUT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'LongMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'Late_avg': {u'generate_derivative': False, u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'sendToInput': u'main', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'DoubleMeaning', u'autoModifiedByDSS': False}, u'role': u'TARGET', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'dep_woy': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'INPUT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'LongMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'Distance': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'INPUT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'LongMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'Dest': {u'missing_impute_with': u'MODE', u'autoReason': u'REJECT_CARDINALITY', u'hash_whole_categories': True, u'nb_bins_hashing': 1048576, u'customHandlingCode': u'', u'category_handling': u'DUMMIFY', u'cumulative_proportion': 0.95, u'sendToInput': u'main', u'state': {u'userModified': False, u'recordedMeaning': u'Text', u'autoModifiedByDSS': False}, u'missing_handling': u'NONE', u'max_nb_categories': 100, u'customProcessorWantsMatrix': False, u'dummy_clip': u'MAX_NB_CATEGORIES', u'role': u'REJECT', u'max_cat_safety': 200, u'min_samples': 10, u'type': u'CATEGORY', u'dummy_drop': u'NONE'}, u'dep_month': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'INPUT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'LongMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'dep_dow': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'INPUT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'LongMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'Late': {u'missing_impute_with': u'MODE', u'customProcessorWantsMatrix': False, u'hash_whole_categories': True, u'nb_bins_hashing': 1048576, u'customHandlingCode': u'', u'category_handling': u'DUMMIFY', u'cumulative_proportion': 0.95, u'sendToInput': u'main', u'state': {u'userModified': False, u'recordedMeaning': u'Boolean', u'autoModifiedByDSS': False}, u'missing_handling': u'NONE', u'max_nb_categories': 5, u'dummy_clip': u'MAX_NB_CATEGORIES', u'role': u'INPUT', u'max_cat_safety': 200, u'min_samples': 10, u'type': u'CATEGORY', u'dummy_drop': u'DROP'}, u'AirTime': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'INPUT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'LongMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}}, u'feature_generation': {u'manual_interactions': {u'interactions': []}, u'pairwise_linear': {u'behavior': u'DISABLED'}, u'categoricals_count_transformer': {u'input_features': [], u'all_features': False, u'behavior': u'DISABLED'}, u'polynomial_combinations': {u'behavior': u'DISABLED'}, u'numericals_clustering': {u'k': 0, u'input_features': [], u'all_features': False, u'behavior': u'DISABLED'}}}
[2021-04-01 23:12:16,139] [5376/MainThread] [INFO] [dataiku.doctor.utils.listener] START -  Loading train set
[2021-04-01 23:12:16,165] [5376/MainThread] [INFO] [root] Reading with dtypes: None
[2021-04-01 23:12:16,165] [5376/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for dep_month: <type 'numpy.float64'> (schema_type=bigint feature_type=NUMERIC feature_role=INPUT)
[2021-04-01 23:12:16,166] [5376/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for dep_woy: <type 'numpy.float64'> (schema_type=bigint feature_type=NUMERIC feature_role=INPUT)
[2021-04-01 23:12:16,166] [5376/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for dep_dow: <type 'numpy.float64'> (schema_type=bigint feature_type=NUMERIC feature_role=INPUT)
[2021-04-01 23:12:16,166] [5376/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for dep_hour: <type 'numpy.float64'> (schema_type=bigint feature_type=NUMERIC feature_role=INPUT)
[2021-04-01 23:12:16,166] [5376/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for UniqueCarrier: str (schema_type=string feature_type=CATEGORY feature_role=INPUT)
[2021-04-01 23:12:16,166] [5376/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for FlightNum: <type 'numpy.float64'> (schema_type=bigint feature_type=NUMERIC feature_role=INPUT)
[2021-04-01 23:12:16,166] [5376/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for AirTime: <type 'numpy.float64'> (schema_type=bigint feature_type=NUMERIC feature_role=INPUT)
[2021-04-01 23:12:16,166] [5376/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for Late: str (schema_type=boolean feature_type=CATEGORY feature_role=INPUT)
[2021-04-01 23:12:16,166] [5376/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for Origin: None (schema_type=string feature_type=CATEGORY feature_role=REJECT)
[2021-04-01 23:12:16,166] [5376/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for Dest: None (schema_type=string feature_type=CATEGORY feature_role=REJECT)
[2021-04-01 23:12:16,166] [5376/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for Distance: <type 'numpy.float64'> (schema_type=bigint feature_type=NUMERIC feature_role=INPUT)
[2021-04-01 23:12:16,166] [5376/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for Late_avg: <type 'numpy.float64'> (schema_type=double feature_type=NUMERIC feature_role=TARGET)
[2021-04-01 23:12:16,166] [5376/MainThread] [INFO] [root] Reading with FIXED dtypes: {u'dep_hour': <type 'numpy.float64'>, u'Late': 'str', u'UniqueCarrier': 'str', u'FlightNum': <type 'numpy.float64'>, u'Late_avg': <type 'numpy.float64'>, u'dep_woy': <type 'numpy.float64'>, u'Distance': <type 'numpy.float64'>, u'dep_month': <type 'numpy.float64'>, u'dep_dow': <type 'numpy.float64'>, u'AirTime': <type 'numpy.float64'>}
[2021-04-01 23:12:16,279] [5376/MainThread] [INFO] [root] Loaded table
[2021-04-01 23:12:16,280] [5376/MainThread] [INFO] [dataiku.doctor.utils]  Coercion done
[2021-04-01 23:12:16,280] [5376/MainThread] [INFO] [dataiku.doctor.utils.split] Loaded train df: shape=(79912,12)
[2021-04-01 23:12:16,280] [5376/MainThread] [INFO] [dataiku.doctor.commands] Train col : dep_month (float64)
[2021-04-01 23:12:16,280] [5376/MainThread] [INFO] [dataiku.doctor.commands] Train col : dep_woy (float64)
[2021-04-01 23:12:16,280] [5376/MainThread] [INFO] [dataiku.doctor.commands] Train col : dep_dow (float64)
[2021-04-01 23:12:16,280] [5376/MainThread] [INFO] [dataiku.doctor.commands] Train col : dep_hour (float64)
[2021-04-01 23:12:16,280] [5376/MainThread] [INFO] [dataiku.doctor.commands] Train col : UniqueCarrier (object)
[2021-04-01 23:12:16,281] [5376/MainThread] [INFO] [dataiku.doctor.commands] Train col : FlightNum (float64)
[2021-04-01 23:12:16,281] [5376/MainThread] [INFO] [dataiku.doctor.commands] Train col : AirTime (float64)
[2021-04-01 23:12:16,281] [5376/MainThread] [INFO] [dataiku.doctor.commands] Train col : Late (object)
[2021-04-01 23:12:16,281] [5376/MainThread] [INFO] [dataiku.doctor.commands] Train col : Origin (object)
[2021-04-01 23:12:16,281] [5376/MainThread] [INFO] [dataiku.doctor.commands] Train col : Dest (object)
[2021-04-01 23:12:16,281] [5376/MainThread] [INFO] [dataiku.doctor.commands] Train col : Distance (float64)
[2021-04-01 23:12:16,281] [5376/MainThread] [INFO] [dataiku.doctor.commands] Train col : Late_avg (float64)
[2021-04-01 23:12:16,281] [5376/MainThread] [INFO] [dataiku.doctor.utils.listener] END -  Loading train set
[2021-04-01 23:12:16,281] [5376/MainThread] [INFO] [dataiku.doctor.utils.listener] START -  Loading test set
[2021-04-01 23:12:16,281] [5376/MainThread] [INFO] [root] Reading with dtypes: None
[2021-04-01 23:12:16,281] [5376/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for dep_month: <type 'numpy.float64'> (schema_type=bigint feature_type=NUMERIC feature_role=INPUT)
[2021-04-01 23:12:16,281] [5376/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for dep_woy: <type 'numpy.float64'> (schema_type=bigint feature_type=NUMERIC feature_role=INPUT)
[2021-04-01 23:12:16,281] [5376/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for dep_dow: <type 'numpy.float64'> (schema_type=bigint feature_type=NUMERIC feature_role=INPUT)
[2021-04-01 23:12:16,282] [5376/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for dep_hour: <type 'numpy.float64'> (schema_type=bigint feature_type=NUMERIC feature_role=INPUT)
[2021-04-01 23:12:16,282] [5376/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for UniqueCarrier: str (schema_type=string feature_type=CATEGORY feature_role=INPUT)
[2021-04-01 23:12:16,282] [5376/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for FlightNum: <type 'numpy.float64'> (schema_type=bigint feature_type=NUMERIC feature_role=INPUT)
[2021-04-01 23:12:16,282] [5376/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for AirTime: <type 'numpy.float64'> (schema_type=bigint feature_type=NUMERIC feature_role=INPUT)
[2021-04-01 23:12:16,282] [5376/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for Late: str (schema_type=boolean feature_type=CATEGORY feature_role=INPUT)
[2021-04-01 23:12:16,282] [5376/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for Origin: None (schema_type=string feature_type=CATEGORY feature_role=REJECT)
[2021-04-01 23:12:16,282] [5376/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for Dest: None (schema_type=string feature_type=CATEGORY feature_role=REJECT)
[2021-04-01 23:12:16,282] [5376/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for Distance: <type 'numpy.float64'> (schema_type=bigint feature_type=NUMERIC feature_role=INPUT)
[2021-04-01 23:12:16,282] [5376/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for Late_avg: <type 'numpy.float64'> (schema_type=double feature_type=NUMERIC feature_role=TARGET)
[2021-04-01 23:12:16,282] [5376/MainThread] [INFO] [root] Reading with FIXED dtypes: {u'dep_hour': <type 'numpy.float64'>, u'Late': 'str', u'UniqueCarrier': 'str', u'FlightNum': <type 'numpy.float64'>, u'Late_avg': <type 'numpy.float64'>, u'dep_woy': <type 'numpy.float64'>, u'Distance': <type 'numpy.float64'>, u'dep_month': <type 'numpy.float64'>, u'dep_dow': <type 'numpy.float64'>, u'AirTime': <type 'numpy.float64'>}
[2021-04-01 23:12:16,312] [5376/MainThread] [INFO] [root] Loaded table
[2021-04-01 23:12:16,313] [5376/MainThread] [INFO] [dataiku.doctor.utils]  Coercion done
[2021-04-01 23:12:16,313] [5376/MainThread] [INFO] [dataiku.doctor.utils.split] Loaded test df: shape=(20088,12)
[2021-04-01 23:12:16,313] [5376/MainThread] [INFO] [dataiku.doctor.utils.listener] END -  Loading test set
[2021-04-01 23:12:16,313] [5376/MainThread] [INFO] [dataiku.doctor.utils.listener] START -  Collecting statistics
[2021-04-01 23:12:16,313] [5376/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at Origin... (type=CATEGORY)
[2021-04-01 23:12:16,313] [5376/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at dep_hour... (type=NUMERIC)
[2021-04-01 23:12:16,313] [5376/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Checking series of type: float64 (isM8=False)
[2021-04-01 23:12:16,317] [5376/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at UniqueCarrier... (type=CATEGORY)
[2021-04-01 23:12:16,329] [5376/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at FlightNum... (type=NUMERIC)
[2021-04-01 23:12:16,329] [5376/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Checking series of type: float64 (isM8=False)
[2021-04-01 23:12:16,332] [5376/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at Late_avg... (type=NUMERIC)
[2021-04-01 23:12:16,332] [5376/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at dep_woy... (type=NUMERIC)
[2021-04-01 23:12:16,332] [5376/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Checking series of type: float64 (isM8=False)
[2021-04-01 23:12:16,334] [5376/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at Distance... (type=NUMERIC)
[2021-04-01 23:12:16,334] [5376/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Checking series of type: float64 (isM8=False)
[2021-04-01 23:12:16,337] [5376/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at Dest... (type=CATEGORY)
[2021-04-01 23:12:16,337] [5376/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at dep_month... (type=NUMERIC)
[2021-04-01 23:12:16,337] [5376/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Checking series of type: float64 (isM8=False)
[2021-04-01 23:12:16,339] [5376/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at dep_dow... (type=NUMERIC)
[2021-04-01 23:12:16,339] [5376/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Checking series of type: float64 (isM8=False)
[2021-04-01 23:12:16,342] [5376/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at Late... (type=CATEGORY)
[2021-04-01 23:12:16,353] [5376/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Looking at AirTime... (type=NUMERIC)
[2021-04-01 23:12:16,353] [5376/MainThread] [INFO] [dataiku.doctor.preprocessing_collector] Checking series of type: float64 (isM8=False)
[2021-04-01 23:12:16,357] [5376/MainThread] [INFO] [dataiku.doctor.utils.listener] END -  Collecting statistics
[2021-04-01 23:12:16,357] [5376/MainThread] [INFO] [dataiku.doctor.multiframe] generating interactions
[2021-04-01 23:12:16,357] [5376/MainThread] [INFO] [dataiku.doctor.multiframe] {u'preprocessingFitSampleSeed': 1337, u'feature_selection_params': {u'custom_params': {u'code': u'# type your code here'}, u'pca_params': {u'variance_proportion': 0.9, u'n_features': 25}, u'random_forest_params': {u'depth': 10, u'n_features': 25, u'n_trees': 30}, u'lasso_params': {u'alpha': [0.01, 0.1, 1.0, 10.0, 100.0], u'cross_validate': True}, u'method': u'NONE', u'correlation_params': {u'max_abs_correlation': 1.0, u'n_features': 25, u'min_abs_correlation': 0.0}}, u'preprocessingFitSampleRatio': 1.0, u'reduce': {u'enabled': False, u'kept_variance': 0.0}, u'skipPreprocessing': False, u'target_remapping': [], u'per_feature': {u'Origin': {u'missing_impute_with': u'MODE', u'autoReason': u'REJECT_CARDINALITY', u'hash_whole_categories': True, u'nb_bins_hashing': 1048576, u'customHandlingCode': u'', u'category_handling': u'DUMMIFY', u'cumulative_proportion': 0.95, u'sendToInput': u'main', u'state': {u'userModified': False, u'recordedMeaning': u'Text', u'autoModifiedByDSS': False}, u'missing_handling': u'NONE', u'max_nb_categories': 100, u'customProcessorWantsMatrix': False, u'dummy_clip': u'MAX_NB_CATEGORIES', u'role': u'REJECT', u'max_cat_safety': 200, u'min_samples': 10, u'type': u'CATEGORY', u'dummy_drop': u'NONE'}, u'dep_hour': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'INPUT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'LongMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'UniqueCarrier': {u'missing_impute_with': u'MODE', u'customProcessorWantsMatrix': False, u'hash_whole_categories': True, u'nb_bins_hashing': 1048576, u'customHandlingCode': u'', u'category_handling': u'DUMMIFY', u'cumulative_proportion': 0.95, u'sendToInput': u'main', u'state': {u'userModified': False, u'recordedMeaning': u'Text', u'autoModifiedByDSS': False}, u'missing_handling': u'NONE', u'max_nb_categories': 5, u'dummy_clip': u'MAX_NB_CATEGORIES', u'role': u'INPUT', u'max_cat_safety': 200, u'min_samples': 10, u'type': u'CATEGORY', u'dummy_drop': u'DROP'}, u'FlightNum': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'INPUT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'LongMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'Late_avg': {u'generate_derivative': False, u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'sendToInput': u'main', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'DoubleMeaning', u'autoModifiedByDSS': False}, u'role': u'TARGET', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'dep_woy': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'INPUT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'LongMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'Distance': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'INPUT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'LongMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'Dest': {u'missing_impute_with': u'MODE', u'autoReason': u'REJECT_CARDINALITY', u'hash_whole_categories': True, u'nb_bins_hashing': 1048576, u'customHandlingCode': u'', u'category_handling': u'DUMMIFY', u'cumulative_proportion': 0.95, u'sendToInput': u'main', u'state': {u'userModified': False, u'recordedMeaning': u'Text', u'autoModifiedByDSS': False}, u'missing_handling': u'NONE', u'max_nb_categories': 100, u'customProcessorWantsMatrix': False, u'dummy_clip': u'MAX_NB_CATEGORIES', u'role': u'REJECT', u'max_cat_safety': 200, u'min_samples': 10, u'type': u'CATEGORY', u'dummy_drop': u'NONE'}, u'dep_month': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'INPUT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'LongMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'dep_dow': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'INPUT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'LongMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}, u'Late': {u'missing_impute_with': u'MODE', u'customProcessorWantsMatrix': False, u'hash_whole_categories': True, u'nb_bins_hashing': 1048576, u'customHandlingCode': u'', u'category_handling': u'DUMMIFY', u'cumulative_proportion': 0.95, u'sendToInput': u'main', u'state': {u'userModified': False, u'recordedMeaning': u'Boolean', u'autoModifiedByDSS': False}, u'missing_handling': u'NONE', u'max_nb_categories': 5, u'dummy_clip': u'MAX_NB_CATEGORIES', u'role': u'INPUT', u'max_cat_safety': 200, u'min_samples': 10, u'type': u'CATEGORY', u'dummy_drop': u'DROP'}, u'AirTime': {u'generate_derivative': False, u'sendToInput': u'main', u'rescaling': u'AVGSTD', u'role': u'INPUT', u'customHandlingCode': u'', u'customProcessorWantsMatrix': False, u'numerical_handling': u'REGULAR', u'binarize_threshold_mode': u'MEDIAN', u'state': {u'userModified': False, u'recordedMeaning': u'LongMeaning', u'autoModifiedByDSS': False}, u'missing_handling': u'IMPUTE', u'binarize_constant_threshold': 0.0, u'quantile_bin_nb_bins': 4, u'missing_impute_with': u'MEAN', u'type': u'NUMERIC', u'impute_constant_value': 0.0}}, u'feature_generation': {u'manual_interactions': {u'interactions': []}, u'pairwise_linear': {u'behavior': u'DISABLED'}, u'categoricals_count_transformer': {u'input_features': [], u'all_features': False, u'behavior': u'DISABLED'}, u'polynomial_combinations': {u'behavior': u'DISABLED'}, u'numericals_clustering': {u'k': 0, u'input_features': [], u'all_features': False, u'behavior': u'DISABLED'}}}
[2021-04-01 23:12:16,357] [5376/MainThread] [INFO] [dataiku.doctor.multiframe] No feature selection to perform
[2021-04-01 23:12:16,357] [5376/MainThread] [INFO] [dataiku.doctor.utils.listener] START -  Preprocessing train set
[2021-04-01 23:12:16,358] [5376/MainThread] [INFO] [dataiku.doctor.multiframe] Set MF index len 79912
[2021-04-01 23:12:16,358] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:RemapValueToOutput
[2021-04-01 23:12:16,358] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:MultipleImputeMissingFromInput
[2021-04-01 23:12:16,358] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] MIMIFI: Imputing with map {u'dep_hour': 13.086732906196817, u'Distance': 724.2432175392933, u'FlightNum': 2149.660088597457, u'dep_woy': 14.544386324957452, u'dep_month': 3.7664931424567025, u'dep_dow': 3.9422364601061166, u'AirTime': 103.37577585343878}
[2021-04-01 23:12:16,363] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:RescalingProcessor2 (dep_hour)
[2021-04-01 23:12:16,363] [5376/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescale dep_hour (avg=13.0867329062 std=4.80188247322 shift=13.0867329062 inv_scale=0.208251660797)
[2021-04-01 23:12:16,368] [5376/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescaled dep_hour (avg=6.69312549233e-16 std=1.0) nulls=0
[2021-04-01 23:12:16,368] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:RescalingProcessor2 (FlightNum)
[2021-04-01 23:12:16,368] [5376/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescale FlightNum (avg=2149.6600886 std=1944.09274632 shift=2149.6600886 inv_scale=0.000514378751678)
[2021-04-01 23:12:16,370] [5376/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescaled FlightNum (avg=3.87140820508e-16 std=1.0) nulls=0
[2021-04-01 23:12:16,370] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:RescalingProcessor2 (dep_woy)
[2021-04-01 23:12:16,371] [5376/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescale dep_woy (avg=14.544386325 std=6.97121381815 shift=14.544386325 inv_scale=0.143447041804)
[2021-04-01 23:12:16,373] [5376/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescaled dep_woy (avg=2.01557300121e-13 std=1.0) nulls=0
[2021-04-01 23:12:16,373] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:RescalingProcessor2 (Distance)
[2021-04-01 23:12:16,373] [5376/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescale Distance (avg=724.243217539 std=561.810411687 shift=724.243217539 inv_scale=0.00177995989252)
[2021-04-01 23:12:16,375] [5376/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescaled Distance (avg=1.16862949168e-16 std=1.0) nulls=0
[2021-04-01 23:12:16,375] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:RescalingProcessor2 (dep_month)
[2021-04-01 23:12:16,376] [5376/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescale dep_month (avg=3.76649314246 std=1.59191042442 shift=3.76649314246 inv_scale=0.628176048512)
[2021-04-01 23:12:16,378] [5376/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescaled dep_month (avg=1.15324419462e-13 std=1.0) nulls=0
[2021-04-01 23:12:16,378] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:RescalingProcessor2 (dep_dow)
[2021-04-01 23:12:16,378] [5376/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescale dep_dow (avg=3.94223646011 std=1.9826725071 shift=3.94223646011 inv_scale=0.50436973147)
[2021-04-01 23:12:16,380] [5376/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescaled dep_dow (avg=1.69635498405e-15 std=1.0) nulls=0
[2021-04-01 23:12:16,380] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:RescalingProcessor2 (AirTime)
[2021-04-01 23:12:16,380] [5376/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescale AirTime (avg=103.375775853 std=68.4857135559 shift=103.375775853 inv_scale=0.0146015854706)
[2021-04-01 23:12:16,382] [5376/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescaled AirTime (avg=5.50103338372e-15 std=1.0) nulls=0
[2021-04-01 23:12:16,382] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:FlushDFBuilder(num_flagonly)
[2021-04-01 23:12:16,382] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:FastSparseDummifyProcessor (UniqueCarrier)
[2021-04-01 23:12:16,397] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] Dummifier: Append a sparse block shape=(79912, 7) nnz=41098
[2021-04-01 23:12:16,397] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:FastSparseDummifyProcessor (Late)
[2021-04-01 23:12:16,411] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] Dummifier: Append a sparse block shape=(79912, 2) nnz=60354
[2021-04-01 23:12:16,412] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:MultipleImputeMissingFromInput
[2021-04-01 23:12:16,412] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] MIMIFI: Imputing with map {}
[2021-04-01 23:12:16,412] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:FlushDFBuilder(cat_flagpresence)
[2021-04-01 23:12:16,412] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:MultipleImputeMissingFromInput
[2021-04-01 23:12:16,412] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] MIMIFI: Imputing with map {}
[2021-04-01 23:12:16,412] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:FlushDFBuilder(interaction)
[2021-04-01 23:12:16,412] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:RealignTarget
[2021-04-01 23:12:16,412] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] Realign target series = (79912,)
[2021-04-01 23:12:16,414] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] After realign target: (79912,)
[2021-04-01 23:12:16,414] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:DropRowsWhereNoTarget
[2021-04-01 23:12:16,415] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] Deleting 0 rows because one of ['target'] is missing
[2021-04-01 23:12:16,415] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] MF before = (79912, 16)
[2021-04-01 23:12:16,415] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] target before = (79912,)
[2021-04-01 23:12:16,420] [5376/MainThread] [INFO] [dataiku.doctor.multiframe] MultiFrame, dropping rows: []
[2021-04-01 23:12:16,443] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] After DRWNT input_df=(79912, 12)
[2021-04-01 23:12:16,443] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] MF after = (79912, 16)
[2021-04-01 23:12:16,443] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] target after = (79912,)
[2021-04-01 23:12:16,443] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:DumpPipelineState
[2021-04-01 23:12:16,443] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] ********* Pipeline state (Before feature selection)
[2021-04-01 23:12:16,443] [5376/MainThread] [DEBUG] [dku.ml.preprocessing]    input_df= (79912, 12) 
[2021-04-01 23:12:16,443] [5376/MainThread] [DEBUG] [dku.ml.preprocessing]    current_mf=(79912, 16) 
[2021-04-01 23:12:16,443] [5376/MainThread] [DEBUG] [dku.ml.preprocessing]    PPR: 
[2021-04-01 23:12:16,443] [5376/MainThread] [DEBUG] [dku.ml.preprocessing]       target = <class 'pandas.core.series.Series'> ((79912,))
[2021-04-01 23:12:16,443] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:EmitCurrentMFAsResult
[2021-04-01 23:12:16,443] [5376/MainThread] [INFO] [dataiku.doctor.multiframe] Set MF index len 79912
[2021-04-01 23:12:16,443] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] FIT/PROCESS WITH Step:DumpPipelineState
[2021-04-01 23:12:16,443] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] ********* Pipeline state (At end)
[2021-04-01 23:12:16,443] [5376/MainThread] [DEBUG] [dku.ml.preprocessing]    input_df= (79912, 12) 
[2021-04-01 23:12:16,443] [5376/MainThread] [DEBUG] [dku.ml.preprocessing]    current_mf=(0, 0) 
[2021-04-01 23:12:16,443] [5376/MainThread] [DEBUG] [dku.ml.preprocessing]    PPR: 
[2021-04-01 23:12:16,443] [5376/MainThread] [DEBUG] [dku.ml.preprocessing]       UNPROCESSED = <class 'pandas.core.frame.DataFrame'> ((79912, 12))
[2021-04-01 23:12:16,443] [5376/MainThread] [DEBUG] [dku.ml.preprocessing]       TRAIN = <class 'dataiku.doctor.multiframe.MultiFrame'> ((79912, 16))
[2021-04-01 23:12:16,443] [5376/MainThread] [DEBUG] [dku.ml.preprocessing]       target = <class 'pandas.core.series.Series'> ((79912,))
[2021-04-01 23:12:16,445] [5376/MainThread] [INFO] [dataiku.doctor.utils.listener] END -  Preprocessing train set
[2021-04-01 23:12:16,445] [5376/MainThread] [INFO] [dataiku.doctor.utils.listener] START -  Preprocessing test set
[2021-04-01 23:12:16,445] [5376/MainThread] [INFO] [dataiku.doctor.multiframe] Set MF index len 20088
[2021-04-01 23:12:16,445] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:RemapValueToOutput
[2021-04-01 23:12:16,445] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:MultipleImputeMissingFromInput
[2021-04-01 23:12:16,445] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] MIMIFI: Imputing with map {u'dep_hour': 13.086732906196817, u'Distance': 724.2432175392933, u'FlightNum': 2149.660088597457, u'dep_woy': 14.544386324957452, u'dep_month': 3.7664931424567025, u'dep_dow': 3.9422364601061166, u'AirTime': 103.37577585343878}
[2021-04-01 23:12:16,447] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:RescalingProcessor2 (dep_hour)
[2021-04-01 23:12:16,447] [5376/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescale dep_hour (avg=13.1375448029 std=4.81152184405 shift=13.0867329062 inv_scale=0.208251660797)
[2021-04-01 23:12:16,448] [5376/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescaled dep_hour (avg=0.0105816618699 std=1.00200741499) nulls=0
[2021-04-01 23:12:16,449] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:RescalingProcessor2 (FlightNum)
[2021-04-01 23:12:16,449] [5376/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescale FlightNum (avg=2129.26144962 std=1938.80005733 shift=2149.6600886 inv_scale=0.000514378751678)
[2021-04-01 23:12:16,450] [5376/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescaled FlightNum (avg=-0.0104926264523 std=0.997277553245) nulls=0
[2021-04-01 23:12:16,450] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:RescalingProcessor2 (dep_woy)
[2021-04-01 23:12:16,450] [5376/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescale dep_woy (avg=14.5089107925 std=6.93786261177 shift=14.544386325 inv_scale=0.143447041804)
[2021-04-01 23:12:16,451] [5376/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescaled dep_woy (avg=-0.00508886018562 std=0.995215868103) nulls=0
[2021-04-01 23:12:16,451] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:RescalingProcessor2 (Distance)
[2021-04-01 23:12:16,452] [5376/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescale Distance (avg=726.248058542 std=562.197650727 shift=724.243217539 inv_scale=0.00177995989252)
[2021-04-01 23:12:16,453] [5376/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescaled Distance (avg=0.00356853657642 std=1.00068926996) nulls=0
[2021-04-01 23:12:16,453] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:RescalingProcessor2 (dep_month)
[2021-04-01 23:12:16,453] [5376/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescale dep_month (avg=3.75985663082 std=1.58189507051 shift=3.76649314246 inv_scale=0.628176048512)
[2021-04-01 23:12:16,454] [5376/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescaled dep_month (avg=-0.00416889765302 std=0.993708594551) nulls=0
[2021-04-01 23:12:16,455] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:RescalingProcessor2 (dep_dow)
[2021-04-01 23:12:16,455] [5376/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescale dep_dow (avg=3.92751891677 std=1.98353439579 shift=3.94223646011 inv_scale=0.50436973147)
[2021-04-01 23:12:16,456] [5376/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescaled dep_dow (avg=-0.00742308338224 std=1.00043471057) nulls=0
[2021-04-01 23:12:16,456] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:RescalingProcessor2 (AirTime)
[2021-04-01 23:12:16,457] [5376/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescale AirTime (avg=103.513590203 std=68.678091524 shift=103.375775853 inv_scale=0.0146015854706)
[2021-04-01 23:12:16,458] [5376/MainThread] [DEBUG] [dku.ml.preprocessing]   Rescaled AirTime (avg=0.00201230800574 std=1.00280902334) nulls=0
[2021-04-01 23:12:16,458] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:FlushDFBuilder(num_flagonly)
[2021-04-01 23:12:16,458] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:FastSparseDummifyProcessor (UniqueCarrier)
[2021-04-01 23:12:16,465] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] Dummifier: Append a sparse block shape=(20088, 7) nnz=10425
[2021-04-01 23:12:16,465] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:FastSparseDummifyProcessor (Late)
[2021-04-01 23:12:16,472] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] Dummifier: Append a sparse block shape=(20088, 2) nnz=15203
[2021-04-01 23:12:16,472] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:MultipleImputeMissingFromInput
[2021-04-01 23:12:16,472] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] MIMIFI: Imputing with map {}
[2021-04-01 23:12:16,472] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:FlushDFBuilder(cat_flagpresence)
[2021-04-01 23:12:16,472] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:MultipleImputeMissingFromInput
[2021-04-01 23:12:16,473] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] MIMIFI: Imputing with map {}
[2021-04-01 23:12:16,473] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:FlushDFBuilder(interaction)
[2021-04-01 23:12:16,473] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:RealignTarget
[2021-04-01 23:12:16,473] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] Realign target series = (20088,)
[2021-04-01 23:12:16,473] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] After realign target: (20088,)
[2021-04-01 23:12:16,473] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:DropRowsWhereNoTarget
[2021-04-01 23:12:16,474] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] Deleting 0 rows because one of ['target'] is missing
[2021-04-01 23:12:16,474] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] MF before = (20088, 16)
[2021-04-01 23:12:16,474] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] target before = (20088,)
[2021-04-01 23:12:16,476] [5376/MainThread] [INFO] [dataiku.doctor.multiframe] MultiFrame, dropping rows: []
[2021-04-01 23:12:16,483] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] After DRWNT input_df=(20088, 12)
[2021-04-01 23:12:16,483] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] MF after = (20088, 16)
[2021-04-01 23:12:16,483] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] target after = (20088,)
[2021-04-01 23:12:16,483] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:DumpPipelineState
[2021-04-01 23:12:16,483] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] ********* Pipeline state (Before feature selection)
[2021-04-01 23:12:16,484] [5376/MainThread] [DEBUG] [dku.ml.preprocessing]    input_df= (20088, 12) 
[2021-04-01 23:12:16,484] [5376/MainThread] [DEBUG] [dku.ml.preprocessing]    current_mf=(20088, 16) 
[2021-04-01 23:12:16,484] [5376/MainThread] [DEBUG] [dku.ml.preprocessing]    PPR: 
[2021-04-01 23:12:16,484] [5376/MainThread] [DEBUG] [dku.ml.preprocessing]       target = <class 'pandas.core.series.Series'> ((20088,))
[2021-04-01 23:12:16,484] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:EmitCurrentMFAsResult
[2021-04-01 23:12:16,484] [5376/MainThread] [INFO] [dataiku.doctor.multiframe] Set MF index len 20088
[2021-04-01 23:12:16,484] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] PROCESS WITH Step:DumpPipelineState
[2021-04-01 23:12:16,484] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] ********* Pipeline state (At end)
[2021-04-01 23:12:16,484] [5376/MainThread] [DEBUG] [dku.ml.preprocessing]    input_df= (20088, 12) 
[2021-04-01 23:12:16,484] [5376/MainThread] [DEBUG] [dku.ml.preprocessing]    current_mf=(0, 0) 
[2021-04-01 23:12:16,484] [5376/MainThread] [DEBUG] [dku.ml.preprocessing]    PPR: 
[2021-04-01 23:12:16,484] [5376/MainThread] [DEBUG] [dku.ml.preprocessing]       UNPROCESSED = <class 'pandas.core.frame.DataFrame'> ((20088, 12))
[2021-04-01 23:12:16,484] [5376/MainThread] [DEBUG] [dku.ml.preprocessing]       TRAIN = <class 'dataiku.doctor.multiframe.MultiFrame'> ((20088, 16))
[2021-04-01 23:12:16,484] [5376/MainThread] [DEBUG] [dku.ml.preprocessing]       target = <class 'pandas.core.series.Series'> ((20088,))
[2021-04-01 23:12:16,484] [5376/MainThread] [INFO] [dataiku.doctor.utils.listener] END -  Preprocessing test set
[2021-04-01 23:12:16,485] [5376/MainThread] [INFO] [dataiku.doctor.utils.listener] START -  Hyperparameter searching
[2021-04-01 23:12:16,506] [5376/MainThread] [DEBUG] [dku.ml.preprocessing] Deleting 0 rows
[2021-04-01 23:12:16,506] [5376/MainThread] [INFO] [dataiku.doctor.multiframe] MultiFrame, dropping rows: []
[2021-04-01 23:12:16,517] [5376/MainThread] [INFO] [dataiku.doctor.prediction.common] prepare multiframe shape=(79912,16) tn=1278592 nnz=660836 fill_ratio=0.52
[2021-04-01 23:12:16,517] [5376/MainThread] [INFO] [dataiku.doctor.prediction.common] too small, using array
[2021-04-01 23:12:16,523] [5376/MainThread] [INFO] [dataiku.doctor.prediction.common] Create CLF from params: {u'computeLearningCurves': False, u'ridge_grid': {u'alpha': {u'values': [0.1, 1.0, 3.0], u'gridMode': u'EXPLICIT', u'randomMode': u'RANGE', u'limit': {u'min': 1e-23}, u'range': {u'scaling': u'LOGARITHMIC', u'max': 3.0, u'nbValues': 3, u'min': 0.1}}, u'alphaMode': u'MANUAL', u'enabled': True}, u'algorithm': u'RIDGE_REGRESSION', u'grid_search_params': {u'nIter': 0, u'nJobs': 4, u'nContainers': 4, u'distributed': False, u'nIterRandom': 3, u'bayesianOptimizer': u'SCIKIT_OPTIMIZE', u'strategy': u'GRID', u'randomized': True, u'shuffleIterations': 1, u'seed': 0, u'mode': u'KFOLD', u'timeout': 0, u'splitRatio': 0.8, u'nFolds': 5, u'stratified': True}, u'autoOptimizeThreshold': True, u'gridLength': 3, u'metrics': {u'customEvaluationMetricNeedsProba': False, u'evaluationMetric': u'R2', u'liftPoint': 0.4, u'costMatrixWeights': {u'fnGain': 0.0, u'tpGain': 1.0, u'tnGain': 0.0, u'fpGain': -0.3}, u'customEvaluationMetricGIB': True}, u'forcedClassifierThreshold': 0.0, u'skipExpensiveReports': False, u'max_ensemble_nodes_serialized': 0, u'pluginAlgoCustomGridSearch': False} for algorithm RIDGE_REGRESSION
[2021-04-01 23:12:16,523] [5376/MainThread] [INFO] [dataiku.doctor.prediction.common] Using K-Fold CV with k=5
[2021-04-01 23:12:16,529] [5376/MainThread] [INFO] [dataiku.doctor.crossval.search_evaluation_monitor] No distributed container configuration is available to run this search
[2021-04-01 23:12:16,535] [5376/MainThread] [INFO] [dataiku.doctor.crossval.search_runner] Fitting 5 folds for each of 3 candidates, totalling 15 fits
[2021-04-01 23:12:16,535] [5376/MainThread] [INFO] [dataiku.doctor.crossval.search_runner] Execute hyperparameter search locally on 4 threads
[2021-04-01 23:12:16,535] [5376/MainThread] [INFO] [dataiku.doctor.crossval.strategies.grid_search_strategy] Running GridSearchStrategy for hyperparameters space: <dataiku.doctor.prediction.common.HyperparametersSpace object at 0x7f57bba54d50>
[2021-04-01 23:12:16,536] [5376/MainThread] [INFO] [dataiku.doctor.crossval.search_evaluator] Schedule evaluation on split 0: {'alpha': 0.1}
[2021-04-01 23:12:16,536] [5376/Thread-2:local-0] [INFO] [dataiku.doctor.distributed.local_worker] Starting worker: local-0
[2021-04-01 23:12:16,536] [5376/Thread-2:local-0] [INFO] [dataiku.doctor.distributed.local_worker] Started worker: local-0
[2021-04-01 23:12:16,536] [5376/Thread-2:local-0] [INFO] [dataiku.doctor.distributed.work_scheduler] Running task...
[2021-04-01 23:12:16,536] [5376/MainThread] [INFO] [dataiku.doctor.crossval.search_evaluator] Schedule evaluation on split 1: {'alpha': 0.1}
[2021-04-01 23:12:16,537] [5376/Thread-2:local-0] [INFO] [dataiku.doctor.crossval.search_context] Fit s=0: alpha=0.1 .......................................................
[2021-04-01 23:12:16,537] [5376/Thread-3:local-1] [INFO] [dataiku.doctor.distributed.local_worker] Starting worker: local-1
[2021-04-01 23:12:16,537] [5376/Thread-3:local-1] [INFO] [dataiku.doctor.distributed.local_worker] Started worker: local-1
[2021-04-01 23:12:16,538] [5376/MainThread] [INFO] [dataiku.doctor.crossval.search_evaluator] Schedule evaluation on split 2: {'alpha': 0.1}
[2021-04-01 23:12:16,538] [5376/Thread-3:local-1] [INFO] [dataiku.doctor.distributed.work_scheduler] Running task...
[2021-04-01 23:12:16,538] [5376/Thread-3:local-1] [INFO] [dataiku.doctor.crossval.search_context] Fit s=1: alpha=0.1 .......................................................
[2021-04-01 23:12:16,539] [5376/Thread-4:local-2] [INFO] [dataiku.doctor.distributed.local_worker] Starting worker: local-2
[2021-04-01 23:12:16,539] [5376/Thread-4:local-2] [INFO] [dataiku.doctor.distributed.local_worker] Started worker: local-2
[2021-04-01 23:12:16,539] [5376/MainThread] [INFO] [dataiku.doctor.crossval.search_evaluator] Schedule evaluation on split 3: {'alpha': 0.1}
[2021-04-01 23:12:16,539] [5376/Thread-4:local-2] [INFO] [dataiku.doctor.distributed.work_scheduler] Running task...
[2021-04-01 23:12:16,540] [5376/Thread-4:local-2] [INFO] [dataiku.doctor.crossval.search_context] Fit s=2: alpha=0.1 .......................................................
[2021-04-01 23:12:16,540] [5376/Thread-5:local-3] [INFO] [dataiku.doctor.distributed.local_worker] Starting worker: local-3
[2021-04-01 23:12:16,541] [5376/Thread-5:local-3] [INFO] [dataiku.doctor.distributed.local_worker] Started worker: local-3
[2021-04-01 23:12:16,541] [5376/Thread-5:local-3] [INFO] [dataiku.doctor.distributed.work_scheduler] Running task...
[2021-04-01 23:12:16,540] [5376/MainThread] [INFO] [dataiku.doctor.crossval.search_evaluator] Schedule evaluation on split 4: {'alpha': 0.1}
[2021-04-01 23:12:16,543] [5376/Thread-5:local-3] [INFO] [dataiku.doctor.crossval.search_context] Fit s=3: alpha=0.1 .......................................................
[2021-04-01 23:12:16,602] [5376/Thread-4:local-2] [INFO] [dataiku.doctor.crossval.search_context] Done s=2: alpha=0.1 (ft=0.0s st=0.0s sc=0.0324189714294, sg=1)
[2021-04-01 23:12:16,604] [5376/Thread-4:local-2] [INFO] [dataiku.doctor.distributed.work_scheduler] Task done
[2021-04-01 23:12:16,604] [5376/MainThread] [INFO] [dataiku.doctor.crossval.search_evaluator] Schedule evaluation on split 0: {'alpha': 1.0}
[2021-04-01 23:12:16,605] [5376/Thread-2:local-0] [INFO] [dataiku.doctor.crossval.search_context] Done s=0: alpha=0.1 (ft=0.0s st=0.0s sc=0.0987544180336, sg=1)
[2021-04-01 23:12:16,606] [5376/Thread-2:local-0] [INFO] [dataiku.doctor.distributed.work_scheduler] Task done
[2021-04-01 23:12:16,606] [5376/Thread-4:local-2] [INFO] [dataiku.doctor.distributed.work_scheduler] Running task...
[2021-04-01 23:12:16,607] [5376/Thread-4:local-2] [INFO] [dataiku.doctor.crossval.search_context] Fit s=4: alpha=0.1 .......................................................
[2021-04-01 23:12:16,608] [5376/Thread-5:local-3] [INFO] [dataiku.doctor.crossval.search_context] Done s=3: alpha=0.1 (ft=0.0s st=0.0s sc=0.0965934774055, sg=1)
[2021-04-01 23:12:16,609] [5376/Thread-5:local-3] [INFO] [dataiku.doctor.distributed.work_scheduler] Task done
[2021-04-01 23:12:16,610] [5376/Thread-3:local-1] [INFO] [dataiku.doctor.crossval.search_context] Done s=1: alpha=0.1 (ft=0.0s st=0.0s sc=0.0336052759112, sg=1)
[2021-04-01 23:12:16,611] [5376/Thread-3:local-1] [INFO] [dataiku.doctor.distributed.work_scheduler] Task done
[2021-04-01 23:12:16,611] [5376/MainThread] [INFO] [dataiku.doctor.crossval.search_evaluator] Schedule evaluation on split 1: {'alpha': 1.0}
[2021-04-01 23:12:16,611] [5376/Thread-2:local-0] [INFO] [dataiku.doctor.distributed.work_scheduler] Running task...
[2021-04-01 23:12:16,611] [5376/Thread-2:local-0] [INFO] [dataiku.doctor.crossval.search_context] Fit s=0: alpha=1.0 .......................................................
[2021-04-01 23:12:16,625] [5376/MainThread] [INFO] [dataiku.doctor.crossval.search_evaluator] Schedule evaluation on split 2: {'alpha': 1.0}
[2021-04-01 23:12:16,625] [5376/Thread-3:local-1] [INFO] [dataiku.doctor.distributed.work_scheduler] Running task...
[2021-04-01 23:12:16,625] [5376/Thread-3:local-1] [INFO] [dataiku.doctor.crossval.search_context] Fit s=1: alpha=1.0 .......................................................
[2021-04-01 23:12:16,629] [5376/MainThread] [INFO] [dataiku.doctor.crossval.search_evaluator] Schedule evaluation on split 3: {'alpha': 1.0}
[2021-04-01 23:12:16,629] [5376/Thread-5:local-3] [INFO] [dataiku.doctor.distributed.work_scheduler] Running task...
[2021-04-01 23:12:16,630] [5376/Thread-5:local-3] [INFO] [dataiku.doctor.crossval.search_context] Fit s=2: alpha=1.0 .......................................................
[2021-04-01 23:12:16,686] [5376/Thread-4:local-2] [INFO] [dataiku.doctor.crossval.search_context] Done s=4: alpha=0.1 (ft=0.0s st=0.0s sc=0.10082519347, sg=1)
[2021-04-01 23:12:16,688] [5376/Thread-4:local-2] [INFO] [dataiku.doctor.distributed.work_scheduler] Task done
[2021-04-01 23:12:16,688] [5376/MainThread] [INFO] [dataiku.doctor.crossval.search_evaluator] Schedule evaluation on split 4: {'alpha': 1.0}
[2021-04-01 23:12:16,688] [5376/Thread-4:local-2] [INFO] [dataiku.doctor.distributed.work_scheduler] Running task...
[2021-04-01 23:12:16,689] [5376/Thread-4:local-2] [INFO] [dataiku.doctor.crossval.search_context] Fit s=3: alpha=1.0 .......................................................
[2021-04-01 23:12:16,693] [5376/Thread-2:local-0] [INFO] [dataiku.doctor.crossval.search_context] Done s=0: alpha=1.0 (ft=0.0s st=0.0s sc=0.0769735426063, sg=1)
[2021-04-01 23:12:16,695] [5376/Thread-2:local-0] [INFO] [dataiku.doctor.distributed.work_scheduler] Task done
[2021-04-01 23:12:16,695] [5376/Thread-2:local-0] [INFO] [dataiku.doctor.distributed.work_scheduler] Running task...
[2021-04-01 23:12:16,696] [5376/Thread-2:local-0] [INFO] [dataiku.doctor.crossval.search_context] Fit s=4: alpha=1.0 .......................................................
[2021-04-01 23:12:16,712] [5376/Thread-5:local-3] [INFO] [dataiku.doctor.crossval.search_context] Done s=2: alpha=1.0 (ft=0.0s st=0.0s sc=0.074442595306, sg=1)
[2021-04-01 23:12:16,716] [5376/Thread-5:local-3] [INFO] [dataiku.doctor.distributed.work_scheduler] Task done
[2021-04-01 23:12:16,723] [5376/Thread-3:local-1] [INFO] [dataiku.doctor.crossval.search_context] Done s=1: alpha=1.0 (ft=0.0s st=0.0s sc=0.0742632287452, sg=1)
[2021-04-01 23:12:16,726] [5376/Thread-3:local-1] [INFO] [dataiku.doctor.distributed.work_scheduler] Task done
[2021-04-01 23:12:16,738] [5376/MainThread] [INFO] [dataiku.doctor.crossval.search_evaluator] Schedule evaluation on split 0: {'alpha': 3.0}
[2021-04-01 23:12:16,738] [5376/MainThread] [INFO] [dataiku.doctor.crossval.search_evaluator] Schedule evaluation on split 1: {'alpha': 3.0}
[2021-04-01 23:12:16,738] [5376/MainThread] [INFO] [dataiku.doctor.crossval.search_evaluator] Schedule evaluation on split 2: {'alpha': 3.0}
[2021-04-01 23:12:16,738] [5376/Thread-3:local-1] [INFO] [dataiku.doctor.distributed.work_scheduler] Running task...
[2021-04-01 23:12:16,739] [5376/Thread-3:local-1] [INFO] [dataiku.doctor.crossval.search_context] Fit s=0: alpha=3.0 .......................................................
[2021-04-01 23:12:16,744] [5376/Thread-5:local-3] [INFO] [dataiku.doctor.distributed.work_scheduler] Running task...
[2021-04-01 23:12:16,744] [5376/Thread-5:local-3] [INFO] [dataiku.doctor.crossval.search_context] Fit s=1: alpha=3.0 .......................................................
[2021-04-01 23:12:16,774] [5376/Thread-4:local-2] [INFO] [dataiku.doctor.crossval.search_context] Done s=3: alpha=1.0 (ft=0.0s st=0.0s sc=0.0802891761552, sg=1)
[2021-04-01 23:12:16,777] [5376/Thread-4:local-2] [INFO] [dataiku.doctor.distributed.work_scheduler] Task done
[2021-04-01 23:12:16,778] [5376/Thread-2:local-0] [INFO] [dataiku.doctor.crossval.search_context] Done s=4: alpha=1.0 (ft=0.0s st=0.0s sc=0.0836404857184, sg=1)
[2021-04-01 23:12:16,779] [5376/Thread-4:local-2] [INFO] [dataiku.doctor.distributed.work_scheduler] Running task...
[2021-04-01 23:12:16,779] [5376/MainThread] [INFO] [dataiku.doctor.crossval.search_evaluator] Schedule evaluation on split 3: {'alpha': 3.0}
[2021-04-01 23:12:16,782] [5376/Thread-2:local-0] [INFO] [dataiku.doctor.distributed.work_scheduler] Task done
[2021-04-01 23:12:16,783] [5376/Thread-4:local-2] [INFO] [dataiku.doctor.crossval.search_context] Fit s=2: alpha=3.0 .......................................................
[2021-04-01 23:12:16,785] [5376/MainThread] [INFO] [dataiku.doctor.crossval.search_evaluator] Schedule evaluation on split 4: {'alpha': 3.0}
[2021-04-01 23:12:16,785] [5376/Thread-2:local-0] [INFO] [dataiku.doctor.distributed.work_scheduler] Running task...
[2021-04-01 23:12:16,787] [5376/Thread-2:local-0] [INFO] [dataiku.doctor.crossval.search_context] Fit s=3: alpha=3.0 .......................................................
[2021-04-01 23:12:16,789] [5376/Thread-5:local-3] [INFO] [dataiku.doctor.crossval.search_context] Done s=1: alpha=3.0 (ft=0.0s st=0.0s sc=0.0462343205014, sg=1)
[2021-04-01 23:12:16,791] [5376/Thread-5:local-3] [INFO] [dataiku.doctor.distributed.work_scheduler] Task done
[2021-04-01 23:12:16,795] [5376/Thread-5:local-3] [INFO] [dataiku.doctor.distributed.work_scheduler] Running task...
[2021-04-01 23:12:16,796] [5376/Thread-5:local-3] [INFO] [dataiku.doctor.crossval.search_context] Fit s=4: alpha=3.0 .......................................................
[2021-04-01 23:12:16,814] [5376/Thread-3:local-1] [INFO] [dataiku.doctor.crossval.search_context] Done s=0: alpha=3.0 (ft=0.0s st=0.0s sc=0.0478850077935, sg=1)
[2021-04-01 23:12:16,817] [5376/Thread-3:local-1] [INFO] [dataiku.doctor.distributed.work_scheduler] Task done
[2021-04-01 23:12:16,842] [5376/Thread-5:local-3] [INFO] [dataiku.doctor.crossval.search_context] Done s=4: alpha=3.0 (ft=0.0s st=0.0s sc=0.0499094974665, sg=1)
[2021-04-01 23:12:16,844] [5376/Thread-5:local-3] [INFO] [dataiku.doctor.distributed.work_scheduler] Task done
[2021-04-01 23:12:16,848] [5376/Thread-4:local-2] [INFO] [dataiku.doctor.crossval.search_context] Done s=2: alpha=3.0 (ft=0.0s st=0.0s sc=0.0464104473712, sg=1)
[2021-04-01 23:12:16,850] [5376/Thread-4:local-2] [INFO] [dataiku.doctor.distributed.work_scheduler] Task done
[2021-04-01 23:12:16,937] [5376/Thread-2:local-0] [INFO] [dataiku.doctor.crossval.search_context] Done s=3: alpha=3.0 (ft=0.0s st=0.0s sc=0.0462812011631, sg=1)
[2021-04-01 23:12:16,939] [5376/Thread-2:local-0] [INFO] [dataiku.doctor.distributed.work_scheduler] Task done
[2021-04-01 23:12:17,537] [5376/Thread-1] [INFO] [dataiku.doctor.crossval.search_runner] Completed search for hyperparameters
[2021-04-01 23:12:17,574] [5376/Thread-1] [INFO] [dataiku.doctor.distributed.work_scheduler] Scheduler has been soft interrupted
[2021-04-01 23:12:17,574] [5376/MainThread] [INFO] [dataiku.doctor.distributed.work_scheduler] Scheduler has been hard interrupted (shutdown)
[2021-04-01 23:12:17,574] [5376/Thread-2:local-0] [INFO] [dataiku.doctor.distributed.local_worker] Stopping worker: local-0
[2021-04-01 23:12:17,574] [5376/Thread-2:local-0] [INFO] [dataiku.doctor.distributed.local_worker] Stopped worker: local-0
[2021-04-01 23:12:17,575] [5376/Thread-5:local-3] [INFO] [dataiku.doctor.distributed.local_worker] Stopping worker: local-3
[2021-04-01 23:12:17,575] [5376/Thread-5:local-3] [INFO] [dataiku.doctor.distributed.local_worker] Stopped worker: local-3
[2021-04-01 23:12:17,575] [5376/Thread-3:local-1] [INFO] [dataiku.doctor.distributed.local_worker] Stopping worker: local-1
[2021-04-01 23:12:17,575] [5376/Thread-3:local-1] [INFO] [dataiku.doctor.distributed.local_worker] Stopped worker: local-1
[2021-04-01 23:12:17,581] [5376/Thread-4:local-2] [ERROR] [dataiku.doctor.distributed.work_scheduler] All workers are dead, interrupt the scheduler
[2021-04-01 23:12:17,581] [5376/Thread-4:local-2] [INFO] [dataiku.doctor.distributed.local_worker] Stopping worker: local-2
[2021-04-01 23:12:17,582] [5376/Thread-4:local-2] [INFO] [dataiku.doctor.distributed.local_worker] Stopped worker: local-2
[2021-04-01 23:12:17,582] [5376/MainThread] [INFO] [dataiku.doctor.crossval.search_runner] Hyperparameter search done, best_parameters being : {'alpha': 1.0}
[2021-04-01 23:12:17,582] [5376/MainThread] [INFO] [dataiku.doctor.utils.listener] END -  Hyperparameter searching
[2021-04-01 23:12:17,621] [5376/MainThread] [INFO] [dataiku.doctor.utils.listener] START -  Fitting model
[2021-04-01 23:12:17,621] [5376/MainThread] [INFO] [dataiku.doctor.prediction.common] Fitting model:
[2021-04-01 23:12:17,622] [5376/MainThread] [INFO] [dataiku.doctor.prediction.common]   Model is: Ridge(alpha=1.0, copy_X=True, fit_intercept=True, max_iter=None,
   normalize=True, random_state=None, solver='auto', tol=0.001)
[2021-04-01 23:12:17,622] [5376/MainThread] [INFO] [dataiku.doctor.prediction.common]   train_X class: <type 'numpy.ndarray'>
[2021-04-01 23:12:17,622] [5376/MainThread] [INFO] [dataiku.doctor.prediction.common]   train_X shape: (79912, 16)
[2021-04-01 23:12:17,622] [5376/MainThread] [INFO] [dataiku.doctor.prediction.common]   train_y shape: (79912,)
[2021-04-01 23:12:17,622] [5376/MainThread] [INFO] [dataiku.doctor.prediction.common]   calibration enabled: a sub-sample of the train data has been saved for calibration
[2021-04-01 23:12:17,646] [5376/MainThread] [INFO] [dataiku.doctor.prediction.regression_fit] Output params are {'resolved': {'ridge': {'alpha': 1.0}, 'skipExpensiveReports': False, 'algorithm': u'RIDGE_REGRESSION'}, 'other': {}}
[2021-04-01 23:12:17,646] [5376/MainThread] [INFO] [dataiku.doctor.prediction.scoring_base] No feature importance in classifier
[2021-04-01 23:12:17,646] [5376/MainThread] [INFO] [dataiku.doctor.utils.listener] END -  Fitting model
[2021-04-01 23:12:17,646] [5376/MainThread] [INFO] [dataiku.doctor.utils.listener] START -  Saving model
[2021-04-01 23:12:17,647] [5376/MainThread] [INFO] [dataiku.doctor.utils.listener] END -  Saving model
[2021-04-01 23:12:17,647] [5376/MainThread] [INFO] [dataiku.doctor.utils.listener] START -  Scoring model
[2021-04-01 23:12:17,648] [5376/MainThread] [INFO] [dataiku.doctor.prediction.regression_scoring] Intrinsic scoring
[2021-04-01 23:12:17,648] [5376/MainThread] [INFO] [dataiku.doctor.prediction.regression_scoring] Extracting rescalers
[2021-04-01 23:12:17,648] [5376/MainThread] [INFO] [dataiku.doctor.prediction.scoring_base] Computing feature importance
[2021-04-01 23:12:17,648] [5376/MainThread] [INFO] [dataiku.doctor.prediction.scoring_base] No feature importance in classifier
[2021-04-01 23:12:17,648] [5376/MainThread] [INFO] [dataiku.doctor.prediction.regression_scoring] Computing regression coeffs
[2021-04-01 23:12:17,665] [5376/MainThread] [INFO] [dataiku.doctor.prediction.scoring_base] Coefs: [ 2.55696719e-01  1.61623301e-03  1.44551141e-03  3.00306230e-03
 -2.33101095e-04  1.67369550e-03 -1.52904442e-03 -1.53850291e-03
 -1.89661253e-02  2.94154035e-03  5.90375398e-03 -1.43326226e-02
  1.45890531e-03  9.24100577e-03  0.00000000e+00 -1.00370527e-02
  0.00000000e+00]
[2021-04-01 23:12:17,665] [5376/MainThread] [INFO] [dataiku.doctor.prediction.scoring_base] Sample variance: 0.004190264764385216
[2021-04-01 23:12:17,670] [5376/MainThread] [INFO] [dataiku.doctor.prediction.scoring_base] Singular variance matrix
[2021-04-01 23:12:17,672] [5376/MainThread] [INFO] [dataiku.doctor.prediction.common] prepare multiframe shape=(20088,16) tn=321408 nnz=166244 fill_ratio=0.52
[2021-04-01 23:12:17,672] [5376/MainThread] [INFO] [dataiku.doctor.prediction.common] too small, using array
[2021-04-01 23:12:17,673] [5376/MainThread] [INFO] [dataiku.doctor.prediction.regression_scoring] Creating predictions on test set
[2021-04-01 23:12:17,682] [5376/MainThread] [INFO] [dataiku.doctor.prediction.regression_scoring] Computing regression performance on [0.2473856  0.2473856  0.2473856  ... 0.23918324 0.26048902 0.23738651]
[2021-04-01 23:12:17,968] [5376/MainThread] [INFO] [dataiku.doctor.diagnostics.diagnostics] Adding diagnostic: <DoctorDiagnostic ML_DIAGNOSTICS_MODEL_CHECK: R2 score is suspiciously low (0.076) - the model is marginally better than a constant mean prediction>
[2021-04-01 23:12:17,968] [5376/MainThread] [INFO] [dataiku.doctor.utils.listener] END -  Scoring model
[2021-04-01 23:12:17,968] [5376/MainThread] [INFO] [root] Reading with dtypes: None
[2021-04-01 23:12:17,969] [5376/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for dep_month: <type 'numpy.float64'> (schema_type=bigint feature_type=NUMERIC feature_role=INPUT)
[2021-04-01 23:12:17,969] [5376/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for dep_woy: <type 'numpy.float64'> (schema_type=bigint feature_type=NUMERIC feature_role=INPUT)
[2021-04-01 23:12:17,969] [5376/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for dep_dow: <type 'numpy.float64'> (schema_type=bigint feature_type=NUMERIC feature_role=INPUT)
[2021-04-01 23:12:17,969] [5376/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for dep_hour: <type 'numpy.float64'> (schema_type=bigint feature_type=NUMERIC feature_role=INPUT)
[2021-04-01 23:12:17,969] [5376/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for UniqueCarrier: str (schema_type=string feature_type=CATEGORY feature_role=INPUT)
[2021-04-01 23:12:17,969] [5376/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for FlightNum: <type 'numpy.float64'> (schema_type=bigint feature_type=NUMERIC feature_role=INPUT)
[2021-04-01 23:12:17,969] [5376/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for AirTime: <type 'numpy.float64'> (schema_type=bigint feature_type=NUMERIC feature_role=INPUT)
[2021-04-01 23:12:17,969] [5376/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for Late: str (schema_type=boolean feature_type=CATEGORY feature_role=INPUT)
[2021-04-01 23:12:17,969] [5376/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for Origin: None (schema_type=string feature_type=CATEGORY feature_role=REJECT)
[2021-04-01 23:12:17,969] [5376/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for Dest: None (schema_type=string feature_type=CATEGORY feature_role=REJECT)
[2021-04-01 23:12:17,969] [5376/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for Distance: <type 'numpy.float64'> (schema_type=bigint feature_type=NUMERIC feature_role=INPUT)
[2021-04-01 23:12:17,969] [5376/MainThread] [INFO] [dataiku.doctor.utils] Computed dtype for Late_avg: <type 'numpy.float64'> (schema_type=double feature_type=NUMERIC feature_role=TARGET)
[2021-04-01 23:12:17,969] [5376/MainThread] [INFO] [root] Reading with FIXED dtypes: {u'dep_hour': <type 'numpy.float64'>, u'Late': 'str', u'UniqueCarrier': 'str', u'FlightNum': <type 'numpy.float64'>, u'Late_avg': <type 'numpy.float64'>, u'dep_woy': <type 'numpy.float64'>, u'Distance': <type 'numpy.float64'>, u'dep_month': <type 'numpy.float64'>, u'dep_dow': <type 'numpy.float64'>, u'AirTime': <type 'numpy.float64'>}
[2021-04-01 23:12:18,010] [5376/MainThread] [INFO] [root] Loaded table
[2021-04-01 23:12:18,012] [5376/MainThread] [INFO] [dataiku.doctor.prediction.background_rows_handler] Building background rows
[2021-04-01 23:12:18,021] [5376/MainThread] [INFO] [dataiku.doctor.prediction.background_rows_handler] Background estimated bias <= 0.000383482381861
[2021-04-01 23:12:18,025] [5376/MainThread] [INFO] [dataiku.doctor.prediction.background_rows_handler] Built background rows with shape=(1000, 12)
[2021-04-01 23:12:18,026] [5376/MainThread] [INFO] [dataiku.doctor.prediction.background_rows_handler] Saving background rows with shape=(1000, 12)
[2021-04-01 23:12:18,033] [5376/MainThread] [INFO] [dataiku.doctor.utils]  Coercion done
[2021-04-01 23:12:18,035] [5376/MainThread] [INFO] [dataiku.doctor.prediction.histogram_handler] Computing histograms
[2021-04-01 23:12:18,070] [5376/MainThread] [INFO] [dataiku.doctor.prediction.histogram_handler] Saving histograms
[2021-04-01 23:12:18,072] [5376/MainThread] [INFO] [dataiku.doctor.prediction.column_importance_handler] Computing column importance
/data/home/dkuacademy/Applications/dataiku-dss-9.0.0/python/dataiku/doctor/prediction/column_importance_handler.py:132: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples,), for example using ravel().
  clf.fit(preprocessed_array[idx], target[idx])
[2021-04-01 23:12:18,348] [5376/MainThread] [INFO] [dataiku.doctor.prediction.column_importance_handler] Saving column importance
[2021/04/01-23:12:18.353] [MRT-67796] [INFO] [dku.block.link.interaction]  - Check result for nullity exceptionIfNull=true result=not null
[2021/04/01-23:12:18.353] [MRT-67796] [INFO] [dku.analysis.prediction]  - Training returned ok
[2021-04-01 23:12:18,353] [5376/MainThread] [INFO] [dataiku.base.socket_block_link] Client closed
[2021/04/01-23:12:18.353] [MRT-67796] [INFO] [dku.ml.versioning]  - Dumping version info {"trainedWithDSSVersion":"9.0.0","trainedWithDSSConfVersion":"9000","trainedWithDSSBackend":"PY_MEMORY","backendCompatibilityVersion":18} in /data/home/dkuacademy/Library/dss-9.0/design/analysis-data/QSDE_SF_TO_FS_V2/oSylVxXB/0IfwLWGQ/sessions/s2/pp2/m1
[2021/04/01-23:12:18.357] [MRT-67796] [INFO] [dku.analysis.ml]  - preTrain={
  "algorithm": "RIDGE_REGRESSION",
  "ridge_grid": {
    "alphaMode": "MANUAL",
    "alpha": {
      "gridMode": "EXPLICIT",
      "randomMode": "RANGE",
      "limit": {
        "min": 1.0E-23
      },
      "values": [
        0.1,
        1.0,
        3.0
      ],
      "range": {
        "min": 0.1,
        "max": 3.0,
        "nbValues": 3,
        "scaling": "LOGARITHMIC"
      }
    },
    "enabled": true
  },
  "max_ensemble_nodes_serialized": 0,
  "metrics": {
    "evaluationMetric": "R2",
    "customEvaluationMetricGIB": true,
    "customEvaluationMetricNeedsProba": false,
    "costMatrixWeights": {
      "tpGain": 1.0,
      "tnGain": 0.0,
      "fpGain": -0.3,
      "fnGain": 0.0
    },
    "liftPoint": 0.4
  },
  "autoOptimizeThreshold": true,
  "forcedClassifierThreshold": 0.0,
  "gridLength": 3,
  "grid_search_params": {
    "mode": "KFOLD",
    "splitRatio": 0.8,
    "shuffleIterations": 1,
    "nFolds": 5,
    "stratified": true,
    "strategy": "GRID",
    "bayesianOptimizer": "SCIKIT_OPTIMIZE",
    "randomized": true,
    "seed": 0,
    "nIter": 0,
    "nIterRandom": 3,
    "timeout": 0,
    "nJobs": 4,
    "distributed": false,
    "nContainers": 4
  },
  "pluginAlgoCustomGridSearch": false,
  "computeLearningCurves": false,
  "skipExpensiveReports": false
}
[2021/04/01-23:12:18.357] [MRT-67796] [INFO] [dku.analysis.ml]  - amp={
  "resolved": {
    "algorithm": "RIDGE_REGRESSION",
    "ridge": {
      "alpha": 1.0
    },
    "skipExpensiveReports": false
  },
  "other": {}
}
[2021/04/01-23:12:18.358] [MRT-67796] [WARN] [dku.kernels]  - Killing kernel python-single-command-kernel
[2021/04/01-23:12:18.358] [MRT-67796] [INFO] [dku.security.process]  - Killing process PID: 5376
[2021/04/01-23:12:18.364] [MRT-67796] [INFO] [dip.tickets]  - Destroying API ticket for analysis-ml-QSDE_SF_TO_FS_V2-f1l8K58 on behalf of tara
[2021/04/01-23:12:18.381] [KNL-python-single-command-kernel-out-67806] [DEBUG] [process]  - StreamToLine: EOF (stream closed)
[2021/04/01-23:12:18.382] [KNL-python-single-command-kernel-err-67807] [DEBUG] [process]  - StreamToLine: EOF (stream closed)
[2021/04/01-23:12:18.384] [KNL-python-single-command-kernel-monitor-67803] [INFO] [dku.kernels]  - Process done with code 143
[2021/04/01-23:12:18.386] [KNL-python-single-command-kernel-monitor-67803] [WARN] [dku.resource]  - stat file for pid 5376 does not exist. Process died?
[2021/04/01-23:12:18.386] [KNL-python-single-command-kernel-monitor-67803] [INFO] [dku.resourceusage]  - Reporting completion of CRU:{"context":{"type":"ANALYSIS_ML_TRAIN","authIdentifier":"tara","projectKey":"QSDE_SF_TO_FS_V2","analysisId":"oSylVxXB","mlTaskId":"0IfwLWGQ","sessionId":"s2"},"type":"LOCAL_PROCESS","id":"DPkaMTEDmvR8Jj5M","startTime":1617311535384,"localProcess":{"pid":5376,"commandName":"/data/home/dkuacademy/Library/dss-9.0/design/bin/python","cpuUserTimeMS":0,"cpuSystemTimeMS":0,"cpuChildrenUserTimeMS":0,"cpuChildrenSystemTimeMS":0,"cpuTotalMS":0,"cpuCurrent":0.0,"vmSizeMB":18,"vmRSSMB":3,"vmHWMMB":3,"vmRSSAnonMB":1,"vmDataMB":1,"vmSizePeakMB":18,"vmRSSPeakMB":3,"vmRSSTotalMBS":0,"majorFaults":0,"childrenMajorFaults":0}}
[2021/04/01-23:12:18.386] [MRT-67796] [INFO] [dku.block.link]  - Closed socket
[2021/04/01-23:12:18.386] [MRT-67796] [INFO] [dku.block.link]  - Closed socket
[2021/04/01-23:12:18.386] [MRT-67796] [INFO] [dku.block.link]  - Closed serverSocket
