模板网站建设公司,静态网站开发技术,wordpress付费主题,数据库和网站建设的论文文章目录 14.4 USDA Food Database#xff08;美国农业部食品数据库#xff09; 14.4 USDA Food Database#xff08;美国农业部食品数据库#xff09;
这个数据是关于食物营养成分的。存储格式是JSON#xff0c;看起来像这样#xff1a;
{id: 21441, 美国农业部食品数据库 14.4 USDA Food Database美国农业部食品数据库
这个数据是关于食物营养成分的。存储格式是JSON看起来像这样
{id: 21441, description: KENTUCKY FRIED CHICKEN, Fried Chicken, EXTRA CRISPY, Wing, meat and skin with breading, tags: [KFC], manufacturer: Kentucky Fried Chicken, group: Fast Foods, portions: [ { amount: 1, unit: wing, with skin, grams: 68.0}...],nutrients: [ { value: 20.8, units: g, description: Protein, group: Composition },...]
} 每种食物都有一系列特征其中有两个listprotions和nutrients。我们必须把这样的数据进行处理方便之后的分析。
这里使用python内建的json模块
import pandas as pd
import numpy as np
import jsonpd.options.display.max_rows 10db json.load(open(../datasets/usda_food/database.json))
len(db)6636db[0].keys()dict_keys([manufacturer, description, group, id, tags, nutrients, portions])db[0][nutrients][0]{description: Protein,group: Composition,units: g,value: 25.18}nutrients pd.DataFrame(db[0][nutrients])
nutrientsdescriptiongroupunitsvalue0ProteinCompositiong25.1801Total lipid (fat)Compositiong29.2002Carbohydrate, by differenceCompositiong3.0603AshOtherg3.2804EnergyEnergykcal376.000...............157SerineAmino Acidsg1.472158CholesterolOthermg93.000159Fatty acids, total saturatedOtherg18.584160Fatty acids, total monounsaturatedOtherg8.275161Fatty acids, total polyunsaturatedOtherg0.830
162 rows × 4 columns
当把由字典组成的list转换为DataFrame的时候我们可以吹创业提取的list部分。这里我们提取食品名群groupID制造商
info_keys [description, group, id, manufacturer]
info pd.DataFrame(db, columnsinfo_keys)
info[:5]descriptiongroupidmanufacturer0Cheese, carawayDairy and Egg Products10081Cheese, cheddarDairy and Egg Products10092Cheese, edamDairy and Egg Products10183Cheese, fetaDairy and Egg Products10194Cheese, mozzarella, part skim milkDairy and Egg Products1028
info.info()class pandas.core.frame.DataFrame
RangeIndex: 6636 entries, 0 to 6635
Data columns (total 4 columns):
description 6636 non-null object
group 6636 non-null object
id 6636 non-null int64
manufacturer 5195 non-null object
dtypes: int64(1), object(3)
memory usage: 207.5 KB我们可以看到食物群的分布使用value_counts:
pd.value_counts(info.group)[:10]Vegetables and Vegetable Products 812
Beef Products 618
Baked Products 496
Breakfast Cereals 403
Legumes and Legume Products 365
Fast Foods 365
Lamb, Veal, and Game Products 345
Sweets 341
Pork Products 328
Fruits and Fruit Juices 328
Name: group, dtype: int64这里我们对所有的nutrient数据做一些分析把每种食物的nutrient部分组合成一个大表格。首先把每个食物的nutrient列表变为DataFrame添加一列为id然后把id添加到DataFrame中接着使用concat联结到一起
# 先创建一个空DataFrame用来保存最后的结果
# 这部分代码运行时间较长请耐心等待
nutrients_all pd.DataFrame()for food in db:nutrients pd.DataFrame(food[nutrients])nutrients[id] food[id]nutrients_all nutrients_all.append(nutrients, ignore_indexTrue)译者虽然作者在书中说了用concat联结在一起但我实际测试后这个concat的方法非常耗时用时几乎是append方法的两倍所以上面的代码中使用了append方法。 一切正常的话出来的效果是这样的
nutrients_alldescriptiongroupunitsvalueid0ProteinCompositiong25.18010081Total lipid (fat)Compositiong29.20010082Carbohydrate, by differenceCompositiong3.06010083AshOtherg3.28010084EnergyEnergykcal376.0001008..................389350Vitamin B-12, addedVitaminsmcg0.00043546389351CholesterolOthermg0.00043546389352Fatty acids, total saturatedOtherg0.07243546389353Fatty acids, total monounsaturatedOtherg0.02843546389354Fatty acids, total polyunsaturatedOtherg0.04143546
389355 rows × 5 columns
这个DataFrame中有一些重复的部分看一下有多少重复的行
nutrients_all.duplicated().sum() # number of duplicates14179把重复的部分去掉
nutrients_all nutrients_all.drop_duplicates()
nutrients_alldescriptiongroupunitsvalueid0ProteinCompositiong25.18010081Total lipid (fat)Compositiong29.20010082Carbohydrate, by differenceCompositiong3.06010083AshOtherg3.28010084EnergyEnergykcal376.0001008..................389350Vitamin B-12, addedVitaminsmcg0.00043546389351CholesterolOthermg0.00043546389352Fatty acids, total saturatedOtherg0.07243546389353Fatty acids, total monounsaturatedOtherg0.02843546389354Fatty acids, total polyunsaturatedOtherg0.04143546
375176 rows × 5 columns
为了与info_keys中的group和descripton区别开我们把列名更改一下
col_mapping {description: food,group: fgroup}info info.rename(columnscol_mapping, copyFalse)
info.info()class pandas.core.frame.DataFrame
RangeIndex: 6636 entries, 0 to 6635
Data columns (total 4 columns):
food 6636 non-null object
fgroup 6636 non-null object
id 6636 non-null int64
manufacturer 5195 non-null object
dtypes: int64(1), object(3)
memory usage: 207.5 KBcol_mapping {description : nutrient,group: nutgroup}nutrients_all nutrients_all.rename(columnscol_mapping, copyFalse)
nutrients_allnutrientnutgroupunitsvalueid0ProteinCompositiong25.18010081Total lipid (fat)Compositiong29.20010082Carbohydrate, by differenceCompositiong3.06010083AshOtherg3.28010084EnergyEnergykcal376.0001008..................389350Vitamin B-12, addedVitaminsmcg0.00043546389351CholesterolOthermg0.00043546389352Fatty acids, total saturatedOtherg0.07243546389353Fatty acids, total monounsaturatedOtherg0.02843546389354Fatty acids, total polyunsaturatedOtherg0.04143546
375176 rows × 5 columns
上面所有步骤结束后我们可以把info和nutrients_all合并merge
ndata pd.merge(nutrients_all, info, onid, howouter)
ndata.info()class pandas.core.frame.DataFrame
Int64Index: 375176 entries, 0 to 375175
Data columns (total 8 columns):
nutrient 375176 non-null object
nutgroup 375176 non-null object
units 375176 non-null object
value 375176 non-null float64
id 375176 non-null int64
food 375176 non-null object
fgroup 375176 non-null object
manufacturer 293054 non-null object
dtypes: float64(1), int64(1), object(6)
memory usage: 25.8 MBndata.iloc[30000]nutrient Glycine
nutgroup Amino Acids
units g
value 0.04
id 6158
food Soup, tomato bisque, canned, condensed
fgroup Soups, Sauces, and Gravies
manufacturer
Name: 30000, dtype: object我们可以对食物群food group和营养类型nutrient type分组后对中位数进行绘图
result ndata.groupby([nutrient, fgroup])[value].quantile(0.5)%matplotlib inlineresult[Zinc, Zn].sort_values().plot(kindbarh, figsize(10, 8))我们还可以找到每一种营养成分含量最多的食物是什么
by_nutrient ndata.groupby([nutgroup, nutrient])get_maximum lambda x: x.loc[x.value.idxmax()]
get_minimum lambda x: x.loc[x.value.idxmin()]max_foods by_nutrient.apply(get_maximum)[[value, food]]# make the food a little smaller
max_foods.food max_foods.food.str[:50]因为得到的DataFrame太大这里只输出Amino Acids(氨基酸)的营养群nutrient group:
max_foods.loc[Amino Acids][food]nutrient
Alanine Gelatins, dry powder, unsweetened
Arginine Seeds, sesame flour, low-fat
Aspartic acid Soy protein isolate
Cystine Seeds, cottonseed flour, low fat (glandless)
Glutamic acid Soy protein isolate...
Serine Soy protein isolate, PROTEIN TECHNOLOGIES INTE...
Threonine Soy protein isolate, PROTEIN TECHNOLOGIES INTE...
Tryptophan Sea lion, Steller, meat with fat (Alaska Native)
Tyrosine Soy protein isolate, PROTEIN TECHNOLOGIES INTE...
Valine Soy protein isolate, PROTEIN TECHNOLOGIES INTE...
Name: food, Length: 19, dtype: object
