# Split DataFrame into validation and train split

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fsplit-dataframe-into-validation-and-train-split%2F16678%3E%5C
**Category:** 🤗Datasets\
**Created:** [April 11, 2022, 7:07am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fsplit-dataframe-into-validation-and-train-split%2F16678 "2022-04-11T07:07:27Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![mwitiderrick](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/mwitiderrick/32/17324_2.png) [@mwitiderrick](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fmwitiderrick)%5C
**Post date:** [April 11, 2022, 7:07am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fsplit-dataframe-into-validation-and-train-split%2F16678%2F1 "2022-04-11T07:07:27Z")

</div>

Is there a way to load data as a Pandas DataFrame and split it into a training and validation split?

I tried this and it didn’t work.

```auto
import datasets
split = (datasets.Split.TRAIN + datasets.Split.TEST).subsplit(datasets.percent[:20])

dataset = Dataset.from_pandas(df,split=split)

```

---

<div class="post-metadata">

**Author:** ![merve](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/merve/32/49809_2.png) [@merve](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fmerve)%5C
**Post date:** [April 11, 2022, 10:54am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fsplit-dataframe-into-validation-and-train-split%2F16678%2F2 "2022-04-11T10:54:56Z")

</div>

Hello Derrick 🤗

So when you import a dataset from pandas you turn it into a `DatasetDict`. When loading datasets with these splits, you need to make sure the dataset has it’s own script that loads those splits. So I’d rather suggest you to split your pandas DataFrames, and then convert them into separate `DatasetDict`s and work on them.  
Still, I’d like to ping @lhoestq if there’s a better solution than mine.

Let me know if this works.

---

<div class="post-metadata">

**Author:** ![lhoestq](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/lhoestq/32/52888_2.png) [@lhoestq](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Flhoestq)%5C
**Post date:** [April 11, 2022, 1:11pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fsplit-dataframe-into-validation-and-train-split%2F16678%2F3 "2022-04-11T13:11:22Z")

</div>

Hi ! You can try

```python
datasets = Dataset.from_pandas(df).train_test_split(test_size=0.2)

train_dataset = datasets["train"]
val_dataset = datasets["test"]

```
