❤️ ❤️ ❤️ 爆：使用ChatGPT+Streamlit快速构建机器学习数据集划分应用程序！！！
ChatGPT 对于 Python 程序员有用吗？特别是我们这些使用 Python 进行数据处理、数据清洗和构建机器学习模型的人？我们试试就知道了。
来自 OpenAI 的 ChatGPT 是什么？假设你已经知道了。网上铺天盖地的宣传呐喊，我想不再需要介绍了。加入您碰巧不知道 ChatGPT 是什么，赶快去查查，不然会被人嘲笑的，然后再回来继续。
与 ChatGPT 的对话可以是有趣的，甚至是有启发性的。但是让我们看看它实际上有多大帮助。
让我们看看 ChatGPT 在根据我们提供的规范编写代码时能做些什么。我们将从简单开始，并从那里开始构建。
一、简单数据集处理脚本
我认为最好从某种数据处理脚本开始。因为我打算稍后让 ChatGPT 使用各种 Python 库编写一些机器学习脚本，所以这似乎是一个合理的起点。
1.目标
首先，我想尝试加载一个数据集; 确保所有数据集值都是数值，或者将它们转换为数值; 检查缺失的值; 并将数据集分割为训练集和测试集。
2.用户提示
Write a Python script that does the following:
- loads a dataset from csv on disk to Pandas DataFrame
- ensures that dataset is entirely numeric; converts string values to numeric and tracks this conversion in a dictionary to reference later
- checks for missing values in the DataFrame and imputes missing values for any record that has one missing value; entirely drops a record that has more than one missing value
- splits the dataset into test and training sets编写一个 Python 脚本，执行以下操作:
- 将磁盘上的 csv 数据集加载到 Pandas DataFrame
- 确保数据集完全是数字; 将字符串值转换为数字，并在字典中跟踪这种转换，以便以后引用
- 检查 DataFrame 中是否缺少值，并为缺少一个值的任何记录计算缺少值; 完全删除一个缺少多个值的记录
- 将数据集分成测试集和训练集
3.GhatGPT返回代码
import pandas as pd
import numpy as np
from sklearn.model_selection import train_test_split# Load the dataset from csv to a Pandas DataFrame
df = pd.read_csv("dataset.csv")# Ensure the dataset is entirely numeric
# Convert string values to numeric and track this conversion in a dictionary to reference later
string_columns = df.columns[df.dtypes == 'object']
string_to_numeric = {}
for column in string_columns:unique_values = df[column].unique()mapping = {value: i for i, value in enumerate(unique_values)}string_to_numeric[column] = mappingdf[column] = df[column].map(mapping)# Check for missing values in the DataFrame
missing_values = df.isna().sum()# Impute missing values for any record that has one missing value; entirely drop a record that has more than one missing value
for column, missing_value_count in missing_values.items():if missing_value_count == 1:df[column].fillna(df[column].mean(), inplace=True)elif missing_value_count > 1:df.dropna(subset=[column], inplace=True)# Split the dataset into test and training sets
train_data, test_data, train_target, test_target = train_test_split(df.drop('target', axis=1), df['target'], test_size=0.2, random_state=0)
注意: 此脚本假定目标变量存储在 DataFrame 中名为 target 的列中。您需要根据数据集的结构修改此代码。
4.结果分析
ChatGPT 生成了一个有用的脚本。它可以加载一个data.csv的数据集，以及训练集/测试集分割的比例，并记录了这两点以及如果需要的话如何更改它们。总的来说，任务完成了。
二、重构我们的代码
1.目标
现在我们已经有了一个执行某些操作的脚本，让我们对它进行一些重构。良好编码的基础是使用函数来组织逻辑代码，促进重用，并确保您不会重复。
2.用户提示
Rewrite the above script to use a series of functions that make sense
重写上面的脚本，使用一系列有意义的函数
3.GhatGPT返回代码
Here’s the script refactored into a series of functions that make sense:
import pandas as pd
import numpy as np
from sklearn.model_selection import train_test_splitdef load_dataset(filepath):"""Loads a dataset from a csv file to a Pandas DataFrame"""return pd.read_csv(filepath)def convert_to_numeric(df):"""Ensures the dataset is entirely numeric and converts string values to numeric"""string_columns = df.columns[df.dtypes == 'object']string_to_numeric = {}for column in string_columns:unique_values = df[column].unique()mapping = {value: i for i, value in enumerate(unique_values)}string_to_numeric[column] = mappingdf[column] = df[column].map(mapping)return df, string_to_numericdef handle_missing_values(df):"""Handles missing values by imputing a mean for records with one missing value, and dropping records with more than one missing value"""missing_values = df.isna().sum()for column, missing_value_count in missing_values.items():if missing_value_count == 1:df[column].fillna(df[column].mean(), inplace=True)elif missing_value_count > 1:df.dropna(subset=[column], inplace=True)return dfdef split_dataset(df, target_column, test_size=0.2, random_state=0):"""Splits the dataset into test and training sets"""train_data, test_data, train_target, test_target = train_test_split(df.drop(target_column, axis=1), df[target_column], test_size=test_size, random_state=random_state)return train_data, test_data, train_target, test_target# Load the dataset from csv to a Pandas DataFrame
df = load_dataset("dataset.csv")# Convert string values to numeric and handle missing values
df, string_to_numeric = convert_to_numeric(df)
df = handle_missing_values(df)# Split the dataset into test and training sets
train_data, test_data, train_target, test_target = split_dataset(df, "target")
Note: This script assumes that the target variable is stored in a column named target in the DataFrame. You’ll need to modify this code based on the structure of your dataset.
在这里，ChatGPT使用了四个函数:
1.load_dataset(filepath) :加载 csv 数据集
2.convert_to_numeric(df) :将字符串值转换为数值。
3.handle_missing_values(df):处理数据集中缺少的值。
4.split_dataset(df, target_column, test_size=0.2, random_state=0) :将数据集划分为测试集和训练集。
测试样例使用这些函数来加载、转换和清理数据集，然后将其拆分为测试集和训练集。
数据集的路径和测试比例是这些函数的输入。
4.结果分析
我们从 ChatGPT 的第一个脚本可以看出一些非常明显的逻辑代码块，可以将这些代码块按功能封装成函数。ChatGPT 做了我期望它做的事情。同样，生成了一个合格的脚本。
三、扩展我们的代码
1.目标
我们的脚本现在看起来好看点了，因为它使用了一些函数。接下来，让我们稍微扩展一下脚本，以便用户可以在运行脚本时指定一些参数。
2.用户提示
Rewrite the above program to solicit the following from the command line in the form of arguments, and use them in the program:
- the path/name of the dataset csv file
- the train/test split size
- the threshold for the number of missing values per record to drop after which to drop entire record, instead of imputing the missing values重写上面的程序，以参数的形式从命令行获取以下内容，并在程序中使用它们:
- 数据集 csv 文件的路径/名称
- 训练集/测试集的分割比例
- 每条记录丢失值数量的阈值，在该阈值之后删除整条记录，而不是输入丢失值
3.GhatGPT返回代码
Here’s the script refactored to solicit the arguments from the command line:
import pandas as pd
import numpy as np
import argparse
from sklearn.model_selection import train_test_splitdef load_dataset(filepath):"""Loads a dataset from a csv file to a Pandas DataFrame"""return pd.read_csv(filepath)def convert_to_numeric(df):"""Ensures the dataset is entirely numeric and converts string values to numeric"""string_columns = df.columns[df.dtypes == 'object']string_to_numeric = {}for column in string_columns:unique_values = df[column].unique()mapping = {value: i for i, value in enumerate(unique_values)}string_to_numeric[column] = mappingdf[column] = df[column].map(mapping)return df, string_to_numericdef handle_missing_values(df, missing_value_threshold):"""Handles missing values by imputing a mean for records with one missing value, and dropping records with more than one missing value"""missing_values = df.isna().sum()for column, missing_value_count in missing_values.items():if missing_value_count <= missing_value_threshold:df[column].fillna(df[column].mean(), inplace=True)elif missing_value_count > missing_value_threshold:df.dropna(subset=[column], inplace=True)return dfdef split_dataset(df, target_column, test_size=0.2, random_state=0):"""Splits the dataset into test and training sets"""train_data, test_data, train_target, test_target = train_test_split(df.drop(target_column, axis=1), df[target_column], test_size=test_size, random_state=random_state)return train_data, test_data, train_target, test_targetif __name__ == '__main__':# Define the command line argumentsparser = argparse.ArgumentParser(description='Handles missing values and splits a dataset into test and training sets')parser.add_argument('filepath', type=str, help='The path/name of the dataset csv file')parser.add_argument('test_size', type=float, help='The test/train split size')parser.add_argument('missing_value_threshold', type=int, help='The number of missing values per record after which to drop the entire record, instead of imputing the missing values')args = parser.parse_args()# Load the dataset from csv to a Pandas DataFramedf = load_dataset(args.filepath)# Convert string values to numeric and handle missing valuesdf, string_to_numeric = convert_to_numeric(df)df = handle_missing_values(df, args.missing_value_threshold)# Split the dataset into test and training setstrain_data, test_data, train_target, test_target = split_dataset(df, df.columns[-1], test_size=args.test_size)
This script can be run from the command line with the following command format:
python script_name.py filepath test_size missing_value_threshold
4.结果分析
老实说，这里不需要太多的分析，因为 ChatGPT 的注释已经很好地解释了它的功能。他竟然告诉我们该怎么执行代码！！！
四、将我们的代码转换为一个 Streamlit 应用程序
1.目标
现在，让假设我们想让这段代码在应用程序中使用。让 ChatGPT 将这些函数包装到 Streamlit 应用程序中。
2.用户提示
Rewrite the above program as a Streamlit app, allowing for the user to provide the same arguments as in the command line args above
将上面的程序重写为 Streamlit 应用程序，允许用户提供与上面命令行 args 中相同的参数
3.GhatGPT返回代码
Here’s the script refactored as a Streamlit app:
import streamlit as st
import pandas as pd
import numpy as np
from sklearn.model_selection import train_test_splitdef load_dataset(filepath):"""Loads a dataset from a csv file to a Pandas DataFrame"""return pd.read_csv(filepath)def convert_to_numeric(df):"""Ensures the dataset is entirely numeric and converts string values to numeric"""string_columns = df.columns[df.dtypes == 'object']string_to_numeric = {}for column in string_columns:unique_values = df[column].unique()mapping = {value: i for i, value in enumerate(unique_values)}string_to_numeric[column] = mappingdf[column] = df[column].map(mapping)return df, string_to_numericdef handle_missing_values(df, missing_value_threshold):"""Handles missing values by imputing a mean for records with one missing value, and dropping records with more than one missing value"""missing_values = df.isna().sum()for column, missing_value_count in missing_values.items():if missing_value_count <= missing_value_threshold:df[column].fillna(df[column].mean(), inplace=True)elif missing_value_count > missing_value_threshold:df.dropna(subset=[column], inplace=True)return dfdef split_dataset(df, target_column, test_size=0.2, random_state=0):"""Splits the dataset into test and training sets"""train_data, test_data, train_target, test_target = train_test_split(df.drop(target_column, axis=1), df[target_column], test_size=test_size, random_state=random_state)return train_data, test_data, train_target, test_targetdef main():"""Main function for the Streamlit app"""st.title("Dataset Preprocessing App")# Get the file path, test size, and missing value threshold from the userfile_path = st.text_input("Enter the path/name of the dataset csv file:")test_size = st.slider("Enter the test/train split size:", 0.0, 1.0, 0.2)missing_value_threshold = st.slider("Enter the number of missing values per record after which to drop the entire record, instead of imputing the missing values:", 0, 100, 1)# Load the dataset and convert string values to numericdf = load_dataset(file_path)df, string_to_numeric = convert_to_numeric(df)# Handle missing valuesdf = handle_missing_values(df, missing_value_threshold)# Split the dataset into test and training setstrain_data, test_data, train_target, test_target = split_dataset(df, df.columns[-1], test_size=test_size)# Show the final processed datast.write("Processed Dataset:")st.write(df)st.write("Train Data:")st.write(train_data)st.write("Train Target:")st.write(train_target)st.write("Test Data:")st.write(test_data)st.write("Test Target:")st.write(test_target)
if name == 'main':main()To run the Streamlit app, you can save the above code in a
.pyfile and run it in the command line with the following command:要运行 Streamlit 应用程序，可以将上面的代码保存在‘.py‘ 文件中，并使用以下命令在命令行中运行它:
streamlit run file_name.py
4.结果分析
它做了它该做的事。ChatGPT 利用我们的功能，建立了一个简单的 Streamlit Web应用。
五、总结
完成以上四步之后，我们检验下代码是否可以正常运行，需要做一下几步操作：
1、安装streamlit 库
pip install streamlit
2、安装 scikit-learn库
pip install scikit-learn
3、开开心心的执行
streamlit run main.py
vscode代码：
运行的web程序：
搞定！NB class ChatGPT！
