做营销的网站建设,鹰潭网站制作,中国企业信用信息公示网查询系统,免费的软件网站前言
上一篇文章#xff0c;讲解了《肥胖风险的多类别预测》机器学习方面的文章#xff0c;主要是多分类算法的运用#xff0c;本文是一个回归的算法#xff0c;本期是2024年4月份的题目《Regression with an Abalone Dataset》即《鲍鱼年龄预测》#xff0c;在此分享高手…前言
上一篇文章讲解了《肥胖风险的多类别预测》机器学习方面的文章主要是多分类算法的运用本文是一个回归的算法本期是2024年4月份的题目《Regression with an Abalone Dataset》即《鲍鱼年龄预测》在此分享高手在这个比赛过程的代码。 由于内容比较多分成两篇来完成上篇着重 讲解数据探索 EDA 方面下篇讲解 建模优化方面。
题目简介
先上一张鲍鱼的图。 这一段话写得非常棒我就直接引用了。 在这个激动人心的探索中我们深入了解鲍鱼生物学的迷人世界。我们的使命利用先进的机器学习技术根据鲍鱼的身体测量结果预测鲍鱼的年龄。 我们的数据集是从对这些海洋软体动物的细致研究和观察中收集的信息宝库。 从外壳尺寸到身体质量我们拥有解开鲍鱼年龄预测秘密所需的所有工具。 但为什么这个预测任务很重要鲍鱼的年龄传统上是通过费力的方法来确定的包括外壳切片和显微镜检查。 ️♂️ 通过利用现成的物理测量我们旨在简化这一过程并提供更快、更高效的替代方案。 当我们开始这段旅程时我们将深入研究数据集探索鲍鱼的不同特征与年龄之间的关系。通过分类和回归技术我们将努力将均方根对数误差RMSLE降至最低并提供准确的预测。 所以准备好让自己沉浸在鲍鱼生物学和预测建模的世界中。用决心和创新让我们一起揭开鲍鱼年龄预测的奥秘✨ 加载库 保证完整性这部分也写上了。 # 加载库
import warnings as wrn
wrn.filterwarnings(ignore, category DeprecationWarning)
wrn.filterwarnings(ignore, category FutureWarning)
wrn.filterwarnings(ignore, category UserWarning) import optuna
import xgboost as xgb
import pandas as pd
import matplotlib.pyplot as plt
import numpy as np
import scipy.stats as stats
import seaborn as sns
import matplotlib.pyplot as plt
from sklearn.model_selection import GroupKFold
from sklearn.metrics import accuracy_score, classification_report, mean_absolute_error
from sklearn.ensemble import RandomForestRegressor
from sklearn.svm import LinearSVC
from sklearn.preprocessing import RobustScaler
from sklearn.pipeline import make_pipeline
from sklearn.decomposition import PCA
from sklearn.model_selection import cross_val_score
from sklearn.metrics import make_scorer, accuracy_score, median_absolute_error
from imblearn.over_sampling import RandomOverSampler
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import LabelEncoder
from sklearn.metrics import mean_squared_error, r2_score
import lightgbm as lgb
import numpy as np
from scipy import stats加载数据
# 加载所有数据
train_data pd.read_csv(/kaggle/input/playground-series-s4e4/train.csv)
test_data pd.read_csv(/kaggle/input/playground-series-s4e4/test.csv)
original_data pd.read_csv(/kaggle/input/abalone-dataset/abalone.csv)这里特别说明一下这个original_data 是原始数据集即来自于UCI。参考链接为https://archive.ics.uci.edu/dataset/1/abalone 查看数据
y train_data[Rings]
id_test test_data[id]train_data train_data.drop([id], axis 1)
test_data test_data.drop([id], axis 1)train_data.columns original_data.columns
test_data.columns original_data.drop([Rings], axis1).columnstrain_data.head()SexLengthDiameterHeightWhole weightShucked weightViscera weightShell weightRings0F0.5500.4300.1500.77150.32850.14650.2400111F0.6300.4900.1451.13000.45800.27650.3200112I0.1600.1100.0250.02100.00550.00300.005063M0.5950.4750.1500.91450.37550.20550.2500104I0.5550.4250.1300.78200.36950.16000.19759
训练集情况
# 查看数据结构num_train_rows, num_train_columns train_data.shapenum_test_rows, num_test_columns test_data.shapenum_original_rows, num_original_columns original_data.shapeprint(Training Data:)
print(fNumber of Rows: {num_train_rows})
print(fNumber of Columns: {num_train_columns}\n)print(Test Data:)
print(fNumber of Rows: {num_test_rows})
print(fNumber of Columns: {num_test_columns}\n)print(Original Data:)
print(fNumber of Rows: {num_original_rows})
print(fNumber of Columns: {num_original_columns})Training Data: Number of Rows: 90615 Number of Columns: 9
Test Data: Number of Rows: 60411 Number of Columns: 8
Original Data: Number of Rows: 4177 Number of Columns: 9
# 查看重复缺失等情况missing_values_train pd.DataFrame({Feature: train_data.columns,[TRAIN] No. of Missing Values: train_data.isnull().sum().values,[TRAIN] % of Missing Values: ((train_data.isnull().sum().values)/len(train_data)*100)})missing_values_test pd.DataFrame({Feature: test_data.columns,[TEST] No.of Missing Values: test_data.isnull().sum().values,[TEST] % of Missing Values: ((test_data.isnull().sum().values)/len(test_data)*100)})missing_values_original pd.DataFrame({Feature: original_data.columns,[ORIGINAL] No.of Missing Values: original_data.isnull().sum().values,[ORIGINAL] % of Missing Values: ((original_data.isnull().sum().values)/len(original_data)*100)})unique_values pd.DataFrame({Feature: train_data.columns,No. of Unique Values[FROM TRAIN]: train_data.nunique().values})feature_types pd.DataFrame({Feature: train_data.columns,DataType: train_data.dtypes})merged_df pd.merge(missing_values_train, missing_values_test, onFeature, howleft)
merged_df pd.merge(merged_df, missing_values_original, onFeature, howleft)
merged_df pd.merge(merged_df, unique_values, onFeature, howleft)
merged_df pd.merge(merged_df, feature_types, onFeature, howleft)merged_dfFeature[TRAIN] No. of Missing Values[TRAIN] % of Missing Values[TEST] No.of Missing Values[TEST] % of Missing Values[ORIGINAL] No.of Missing Values[ORIGINAL] % of Missing ValuesNo. of Unique Values[FROM TRAIN]DataType0Sex00.00.00.000.03object1Length00.00.00.000.0157float642Diameter00.00.00.000.0126float643Height00.00.00.000.090float644Whole weight00.00.00.000.03175float645Shucked weight00.00.00.000.01799float646Viscera weight00.00.00.000.0979float647Shell weight00.00.00.000.01129float648Rings00.0NaNNaN00.028int64
数据无重复、无缺失
train_data.describe().Tcountmeanstdmin25%50%75%maxLength90615.00.5170980.1182170.07500.44500.54500.60000.8150Diameter90615.00.4016790.0980260.05500.34500.42500.47000.6500Height90615.00.1354640.0380080.00000.11000.14000.16001.1300Whole weight90615.00.7890350.4576710.00200.41900.79951.06752.8255Shucked weight90615.00.3407780.2044280.00100.17750.33000.46301.4880Viscera weight90615.00.1694220.1009090.00050.08650.16600.23250.7600Shell weight90615.00.2258980.1302030.00150.12000.22500.30501.0050Rings90615.09.6967943.1762211.00008.00009.000011.000029.0000
数据集特征说明
列名完整含义详细说明‘id’id鲍鱼的唯一号‘Sex’性别指示鲍鱼性别的分类变量。可能的值为M男性、F女性和I婴儿‘Length’长度表示最长外壳测量值的连续变量单位为毫米。‘Diameter’直径表示外壳直径的连续变量垂直于长度测量单位为毫米。‘Height’高度表示鲍鱼高度的连续变量鲍鱼壳中有肉单位为毫米。‘Whole_weight’整体重量表示鲍鱼整体重量的连续变量单位为克。‘Shucked_weight’去皮重量表示肉去皮后重量的连续变量单位为克。‘Viscera_weight’内脏重量表示肠道重量出血后的连续变量单位为克。‘Shell_weight’贝壳重量表示贝壳重量干燥后的连续变量单位为克。‘Rings’环以年为单位表示鲍鱼年龄的整数变量。1.5的值表示实际年龄以年为单位。 该数据集为大量鲍鱼标本提供了一套全面的物理测量数据。包括分类变量和连续变量在内的各种特征使该数据集非常适合各种预测建模任务特别是基于这些容易获得的物理特征预测鲍鱼年龄。目标变量Rings表示鲍鱼壳中的环数这是动物年龄的代表。预测鲍鱼的年龄是一项具有挑战性的任务因为传统的环数方法既耗时又乏味。因此该数据集提供了一个机会可以探索使用可用的物理测量来估计鲍鱼年龄的替代、更有效的方法。 数据探索 EDA
numerical_variables [Length, Diameter, Height, Whole weight, Shucked weight, Viscera weight, Shell weight]
target_variable Rings
categorical_variables [Sex]特征分为数值型和分类型本题 数据结构相对比较简单
数值型特征分布图 # 分析数值型特征# Define a custom color palette
custom_palette [#28ae80, #d3eb0c, #ff9a0b
]# Add Dataset column to distinguish between train and test data
train_data[Dataset] Train
test_data[Dataset] Test
original_data[Dataset] Originalvariables [col for col in train_data.columns if col in numerical_variables]# Function to create and display a row of plots for a single variable
def create_variable_plots(variable):sns.set_style(whitegrid)fig, axes plt.subplots(1, 2, figsize(12, 4))# Box plotplt.subplot(1, 2, 1)if variable Whole weight.1 or variable Whole weight.2:sns.boxplot(datapd.concat([train_data, test_data]), xvariable, yDataset, palettecustom_palette)else:sns.boxplot(datapd.concat([train_data, test_data, original_data.dropna()]), xvariable, yDataset, palettecustom_palette)plt.xlabel(variable)plt.title(fBox Plot for {variable}, fontweightbold, fontfamilyserif)# Separate Histogramsplt.subplot(1, 2, 2)if variable Whole weight.1 or variable Whole weight.2:sns.histplot(datatrain_data, xvariable, colorcustom_palette[0], kdeTrue, bins30, labelTrain)sns.histplot(datatest_data, xvariable, colorcustom_palette[1], kdeTrue, bins30, labelTest)else:sns.histplot(datatrain_data, xvariable, colorcustom_palette[0], kdeTrue, bins30, labelTrain)sns.histplot(datatest_data, xvariable, colorcustom_palette[1], kdeTrue, bins30, labelTest)sns.histplot(dataoriginal_data.dropna(), xvariable, colorcustom_palette[2], kdeTrue, bins30, labelOriginal)plt.xlabel(variable)plt.ylabel(Frequency)plt.title(fHistogram for {variable} [TRAIN, TEST ORIGINAL], fontweightbold, fontfamilyserif)plt.legend()# Adjust spacing between subplotsplt.tight_layout()# Show the plotsplt.show()# Perform univariate analysis for each variable
for variable in variables:create_variable_plots(variable)# Drop the Dataset column after analysis
train_data.drop(Dataset, axis1, inplaceTrue)
test_data.drop(Dataset, axis1, inplaceTrue)
original_data.drop(Dataset, axis1, inplaceTrue)分类型特征分布图
# 分析分类型特征# Define a custom color palette for categorical features
categorical_palette [#33638d, #28ae80, #d3eb0c, #ff9a0b]# List of categorical variables
categorical_variables [col for col in categorical_variables]# Function to create and display a row of plots for a single categorical variable
def create_categorical_plots(variable):sns.set_style(whitegrid)fig, axes plt.subplots(1, 2, figsize(12, 4))# Pie Chartplt.subplot(1, 2, 1)train_data[variable].value_counts().plot.pie(autopct%1.1f%%, colorscategorical_palette, wedgepropsdict(width0.3), startangle140)plt.title(fPie Chart for {variable}, fontweightbold, fontfamilyserif)# Bar Graphplt.subplot(1, 2, 2)sns.countplot(datapd.concat([train_data, test_data, original_data.dropna()]), xvariable, palettecategorical_palette)plt.xlabel(variable)plt.ylabel(Count)plt.title(fBar Graph for {variable} [TRAIN, TEST ORIGINAL], fontweightbold, fontfamilyserif)# Adjust spacing between subplotsplt.tight_layout()# Show the plotsplt.show()# Perform univariate analysis for each categorical variable
for variable in categorical_variables:create_categorical_plots(variable)目标特征分布图
# 分析目标特征target_palette [#3b528b,#1fa088,#9ee742,#d6f015,#e0d20a,#ffaa08,#ff522f,#d7043d]fig, axes plt.subplots(1, 2, figsize(12, 4))# Box plot
plt.subplot(1, 2, 1)
sns.boxplot(datatrain_data, xtarget_variable, palettetarget_palette)
plt.xlabel(variable)
plt.title(fBox Plot for {target_variable} [TRAIN], fontweightbold, fontfamilyserif)# Separate Histograms
plt.subplot(1, 2, 2)
sns.histplot(datatrain_data, xtarget_variable, colortarget_palette[1], kdeTrue, bins30, labelTrain)
plt.xlabel(variable)
plt.ylabel(Frequency)
plt.title(fHistogram for {target_variable} [TRAIN], fontweightbold, fontfamilyserif)
plt.legend()# Adjust spacing between subplots
plt.tight_layout()# Show the plots
plt.show()数据集“环”表示鲍鱼壳中的环的数量这是动物年龄的代表。该变量的值范围从1到29根据具体问题和建模方法可以将其视为连续变量和分类变量。
连续变量法
由于“环”变量代表鲍鱼的年龄因此可以将其视为连续变量。这意味着这项任务可以作为一个回归问题来处理其中的目标是根据可用的物理测量来预测每只鲍鱼的确切环数年龄。可以使用线性回归和决策树等建模技术来学习特征与连续“环”目标之间的关系。
分类变量法
或者“环”变量可以被视为一个分类变量其中不同的值1到29代表不同的年龄组或类别。在这种情况下该任务可以被定义为一个分类问题其中的目标是根据物理测量来预测每只鲍鱼的年龄组类别。可以使用逻辑回归、决策树或随机森林等建模技术来学习特征和分类“环”目标之间的映射。
连续变量法或分类变量法之间的选择取决于问题的具体要求和模型的预期用途。如果目标是预测鲍鱼的确切年龄那么连续变量法可能更适合。然而如果重点是将鲍鱼分为不同的年龄组那么分类变量方法可能更合适。
我将探索这两种方法并比较它们的性能以确定为给定问题建模“环”变量的最佳方式。
偏度条形图
# 测量和比较所有特征之间的偏度并使用条形图进行比较skew_merged pd.DataFrame(data train_data[numerical_variables].skew(), columns [Skewness])
skew_sorted skew_merged.sort_values(ascending False, by Skewness)sns.barplot(xskew_sorted.Skewness, yskew_sorted.index, paletteviridis)
plt.title(Skewness in Explanatory Variables, fontweightbold, fontfamilyserif)
plt.xlabel(Value)
plt.ylabel(Skewness)
plt.show()相关性热力图
# 计算相关性的热力图
corr_train train_data[numerical_variables].corr()
corr_test test_data[numerical_variables].corr()# Create masks for the upper triangle
mask_train np.triu(np.ones_like(corr_train, dtypebool))
mask_test np.triu(np.ones_like(corr_test, dtypebool))# Set the text size and rotation
annot_kws {size: 8, rotation: 45}# Generate heatmaps for train_data
plt.figure(figsize(18, 7))
plt.subplot(1, 2, 1)
ax_train sns.heatmap(corr_train, maskmask_train, cmapviridis, annotTrue,squareTrue, linewidths.5, xticklabels1, yticklabels1, annot_kwsannot_kws)
plt.title(Correlation Heatmap - Train Data, fontweightbold, fontfamilyserif)# Generate heatmaps for test_data
plt.subplot(1, 2, 2)
ax_test sns.heatmap(corr_test, maskmask_test, cmapviridis, annotTrue,squareTrue, linewidths.5, xticklabels1, yticklabels1, annot_kwsannot_kws)
plt.title(Correlation Heatmap - Test Data, fontweightbold, fontfamilyserif)# Adjust layout
plt.tight_layout()# Show the plots
plt.show()特征分布散点图
fig, axes plt.subplots(2, 4, figsize(16, 8))
axes axes.flatten()for i, feature in enumerate(numerical_variables):axes[i].scatter(train_data[feature], train_data[Rings], s5)axes[i].set_xlabel(feature)axes[i].set_ylabel(Rings)axes[i].set_title(fRings vs {feature})plt.tight_layout()
plt.show() 散点图显示物理测量值长度、直径、高度、重量与代表鲍鱼年龄的环数之间存在正相关关系。这表明更大、更重的鲍鱼往往更老。
为了量化关系的强度让我们计算“环”和连续特征之间的Pearson相关系数
相关系数
correlations train_data[numerical_variables [Rings]].corr()
print(correlations[Rings])Length 0.623786 Diameter 0.636832 Height 0.665772 Whole weight 0.617274 Shucked weight 0.515067 Viscera weight 0.588954 Shell weight 0.694766 Rings 1.000000 Name: Rings, dtype: float64
相关系数表明所有物理测量值与环的数量都具有中等强度的正相关性其中壳体重量具有最高的相关性0.694766。
性别雄性、雌性和幼年对鲍鱼的年龄分布是否存在差异
性别对年龄的影响
# Bar plot to show the distribution of Rings for each Sex category
plt.figure(figsize(8, 6))
train_data.groupby(Sex)[Rings].mean().plot(kindbar)
plt.xlabel(Sex)
plt.ylabel(Mean Rings)
plt.title(Mean Rings by Sex)
plt.show()条形图按性别划分的平均环
条形图显示了每个性别类别的戒指平均数量。 我们可以看到雌性鲍鱼的平均环数高于雄性和幼年鲍鱼。
# Bar plot to show the distribution of Rings for each Sex category
plt.figure(figsize(8, 6))
train_data.groupby(Sex)[Rings].mean().plot(kindbar)
plt.xlabel(Sex)
plt.ylabel(Mean Rings)
plt.title(Mean Rings by Sex)
plt.show()盒图按性别划分的环分布
盒图提供了《指环王》在性别类别中分布的更详细比较。 该图显示雌性鲍鱼的环的中位数和四分位间距IQR略高于雄性和幼年鲍鱼。 对于所有类别似乎也存在一些具有非常高环值的异常值。
建模
准备数据
dd{M:0,F:1,I:2}
train_data[Sex] train_data[Sex].map(dd)
text_data[Sex] test_data[Sex].map(dd)
Xtrain.drop([Rings],axis1)加载各类算法
from sklearn.linear_model import LogisticRegression,LinearRegression
from sklearn.svm import SVC
from sklearn.discriminant_analysis import LinearDiscriminantAnalysis
from sklearn.naive_bayes import GaussianNB, BernoulliNB
from sklearn.neighbors import KNeighborsClassifier
from sklearn.gaussian_process import GaussianProcessClassifier
from sklearn.ensemble import RandomForestRegressor, ExtraTreesClassifier, GradientBoostingClassifier
from xgboost import XGBClassifier
from lightgbm import LGBMClassifier
from catboost import CatBoostClassifier
from sklearn.ensemble import HistGradientBoostingClassifier
from xgboost import XGBRegressor
from lightgbm import LGBMRegressor
from catboost import CatBoostRegressor
from sklearn.ensemble import RandomForestClassifier, ExtraTreesClassifier
from xgboost import XGBClassifier
from lightgbm import LGBMClassifier# Set a seed for reproducibility
seed 42# Initialize all the classification models in the requested format
log_reg LogisticRegression(random_stateseed, max_iter1000000)
svc SVC(random_stateseed, probabilityTrue)
lda LinearDiscriminantAnalysis()
gnb GaussianNB()
bnb BernoulliNB()
knn KNeighborsClassifier()
gauss GaussianProcessClassifier(random_stateseed)
rf RandomForestRegressor(random_stateseed)
et ExtraTreesClassifier(random_stateseed)
xgb XGBRegressor(random_stateseed)
lgb LGBMRegressor(random_stateseed, verbosity0)
dart LGBMClassifier(random_stateseed, boosting_typedart)
cb CatBoostRegressor(random_stateseed, verbose0)
gb GradientBoostingClassifier(random_stateseed)
hgb HistGradientBoostingClassifier(random_stateseed)
lr LinearRegression()
rf_classifier RandomForestClassifier(random_stateseed)
et_classifier ExtraTreesClassifier(random_stateseed)
xgb_classifier XGBClassifier(random_stateseed)
lgb_classifier LGBMClassifier(random_stateseed, verbosity0)
cb_classifier CatBoostClassifier(random_stateseed, verbose0)选出合适算法
import warnings
# Ignore all warnings
warnings.filterwarnings(ignore)from sklearn.metrics import mean_squared_log_error
import numpy as npdef rmsle(y_true, y_pred):return np.sqrt(mean_squared_log_error(y_true, y_pred))# Fit the models and calculate RMSLE
models [xgb, lgb, cb, rf]
rmsle_scores []for model in models:X_train, X_test, y_train, y_test train_test_split(X,y,test_size0.2,random_state42)model.fit(X_train, y_train)y_pred model.predict(X_test)score rmsle(y_test, y_pred)rmsle_scores.append(score)# Create a DataFrame with the RMSLE scores
rmsle_df pd.DataFrame(datarmsle_scores, columns[RMSLE])
rmsle_df.index [xgb, lgb, cb, rf]
rmsle_df rmsle_df.sort_values(byRMSLE, ascendingFalse)# Create a bar graph using Seaborn
plt.figure(figsize(12, 6))
barplot sns.barplot(xRMSLE, yrmsle_df.index, datarmsle_df, paletteviridis)# Add values on the bars with larger and bolder font
for index, value in enumerate(rmsle_df[RMSLE]):barplot.text(value 0.001, index, str(round(value, 5)), haleft, vacenter, fontsize12, fontweightbold)plt.title(Models RMSLE Scores, fontweightbold, fontfamilyserif)
plt.xlabel(RMSLE)
plt.ylabel(Models)
plt.show()小结
通上述EDA和Modeling 后对数值结果有了一个初步的了解。不同的模型的结果情况有了一个最基础的 Baseline 如何提高比赛成绩将在下一篇做详细的说明。
