diff --git a/.history/index_20231115215500.html b/.history/index_20231115215500.html new file mode 100644 index 0000000..22b8ca2 --- /dev/null +++ b/.history/index_20231115215500.html @@ -0,0 +1,608 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + TableLlama: Towards Open Large Generalist Models for Tables + + + + + + + + + + + + + + + + + + + + + + +
+
+
+
+
+ +
+ Icon +

TableLlama: Towards Open Large Generalist Models for Tables

+
+ +
+ + + 1Tianshu Zhang*, + + 2Xiang Yue, + + 1Yifei Li, + + 1Huan Sun + +
+ +
+ + 1The Ohio State University, + 2IN.AI + +
+ zhang.11535@osu.edu + , sun.397@osu.edu + +
+ + +
+
+ + + + + + + + + + + Code + + + + + + + + + + arXiv + + +
+
+
+
+
+
+
+ + + +
+
+
+
+
+

Abstract

+
+

+ Semi-structured tables are ubiquitous. There has been a variety of tasks that aim to automatically interpret, augment, and query tables. Current methods often require pretraining on tables or special model architecture design, are restricted to specific table types, or have simplifying assumptions about tables and tasks. This paper makes the first step towards developing open-source large language models (LLMs) as generalists for a diversity of table-based tasks. Towards that end, we construct TableInstruct, a new dataset with a variety of realistic tables and tasks, for instruction tuning and evaluating LLMs. We further develop the first open-source generalist model for tables, TableLlama, by fine-tuning Llama 2 (7B) with LongLoRA to address the long context challenge. We experiment under both in-domain setting and out-of-domain setting. On 7 out of 8 in-domain tasks, TableLlama achieves com- parable or better performance than the SOTA for each task, despite the latter often has task-specific design. On 6 out-of-domain datasets, it achieves 6-48 absolute point gains compared with the base model, showing that training on TableInstruct enhances the model’s generalizability. We will open-source our dataset and trained model to boost future work on developing open generalist models for tables. +

+
+
+
+
+
+
+ + + +
+
+
+
+
+
+ + An overview of TableInstruct and 🦙TableLlama +

+ Figure 1: An overview of TableInstruct and TableLlama. TableInstruct includes a wide variety of realistic tables and tasks with instructions. We make the first step towards developing open-source generalist models for tables with TableInstruct and TableLlama. +

+
+
+
+
+
+
+ + + +
+
+
+
+
+
+ + The hybrid instruction tuning of 🦣MAmmoTH +

+ Figure 2: Illustration of three exemplary tasks: (a) Column type annotation. This task is to annotate the selected column with the correct semantic types. (b) Row population. This task is to populate rows given table metadata and partial row entities. (c) Hierarchical table QA. For subfigures (a) and (b), we mark candidates with red color in the + "task instruction" part. The candidate set size can be hundreds to thousands in TableInstruct. +

+
+
+
+
+
+
+ + +
+
+
+
+
+

Our Dataset: TableInstruct

+
+ +
+
+
+
+
+
+ +
+
+
+
+
+

In-domain Evaluation:

+
+ +
+
+
+
+
+
+ + +
+
+
+
+
+

Out-of-domain Evaluation:

+
+ +
+
+
+
+
+
+ + + + +
+
+

Reference

+ Please kindly cite our paper if you use our code, data, models or results: +

+ 
@misc{zhang2023tablellama,
+  title={TableLlama: Towards Open Large Generalist Models for Tables}, 
+  author={Tianshu Zhang and Xiang Yue and Yifei Li and Huan Sun},
+  year={2023},
+  eprint={2311.09206},
+  archivePrefix={arXiv},
+  primaryClass={cs.CL}
+}
+
+
+
+ + + + + + + + + + + + + diff --git a/.history/index_20231115220627.html b/.history/index_20231115220627.html new file mode 100644 index 0000000..fc36733 --- /dev/null +++ b/.history/index_20231115220627.html @@ -0,0 +1,608 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + TableLlama: Towards Open Large Generalist Models for Tables + + + + + + + + + + + + + + + + + + + + + + +
+
+
+
+
+ +
+ Icon +

TableLlama: Towards Open Large Generalist Models for Tables

+
+ +
+ + + 1Tianshu Zhang*, + + 2Xiang Yue, + + 1Yifei Li, + + 1Huan Sun + +
+ +
+ + 1The Ohio State University, + 2IN.AI + +
+ zhang.11535@osu.edu + , sun.397@osu.edu + +
+ + +
+
+ + + + + + + + + + + Code + + + + + + + + + + arXiv + + +
+
+
+
+
+
+
+ + + +
+
+
+
+
+

Abstract

+
+

+ Semi-structured tables are ubiquitous. There has been a variety of tasks that aim to automatically interpret, augment, and query tables. Current methods often require pretraining on tables or special model architecture design, are restricted to specific table types, or have simplifying assumptions about tables and tasks. This paper makes the first step towards developing open-source large language models (LLMs) as generalists for a diversity of table-based tasks. Towards that end, we construct TableInstruct, a new dataset with a variety of realistic tables and tasks, for instruction tuning and evaluating LLMs. We further develop the first open-source generalist model for tables, TableLlama, by fine-tuning Llama 2 (7B) with LongLoRA to address the long context challenge. We experiment under both in-domain setting and out-of-domain setting. On 7 out of 8 in-domain tasks, TableLlama achieves com- parable or better performance than the SOTA for each task, despite the latter often has task-specific design. On 6 out-of-domain datasets, it achieves 6-48 absolute point gains compared with the base model, showing that training on TableInstruct enhances the model’s generalizability. We will open-source our dataset and trained model to boost future work on developing open generalist models for tables. +

+
+
+
+
+
+
+ + + +
+
+
+
+
+
+ + An overview of TableInstruct and 🦙TableLlama +

+ Figure 1: An overview of TableInstruct and TableLlama. TableInstruct includes a wide variety of realistic tables and tasks with instructions. We make the first step towards developing open-source generalist models for tables with TableInstruct and TableLlama. +

+
+
+
+
+
+
+ + + +
+
+
+
+
+
+ + The hybrid instruction tuning of 🦣MAmmoTH +

+ Figure 2: Illustration of three exemplary tasks: (a) Column type annotation. This task is to annotate the selected column with the correct semantic types. (b) Row population. This task is to populate rows given table metadata and partial row entities. (c) Hierarchical table QA. For subfigures (a) and (b), we mark candidates with red color in the + "task instruction" part. The candidate set size can be hundreds to thousands in TableInstruct. +

+
+
+
+
+
+
+ + +
+
+
+
+
+

Our Dataset: TableInstruct

+
+ +
+
+
+
+
+
+ +
+
+
+
+
+

In-domain Evaluation:

+
+ +
+
+
+
+
+
+ + +
+
+
+
+
+

Out-of-domain Evaluation:

+
+ +
+
+
+
+
+
+ + + + +
+
+

Reference

+ Please kindly cite our paper if you use our code, data, models or results: +

+ 
@misc{zhang2023tablellama,
+  title={TableLlama: Towards Open Large Generalist Models for Tables}, 
+  author={Tianshu Zhang and Xiang Yue and Yifei Li and Huan Sun},
+  year={2023},
+  eprint={2311.09206},
+  archivePrefix={arXiv},
+  primaryClass={cs.CL}
+}
+
+
+
+ + + + + + + + + + + + + diff --git a/.history/index_20231115220644.html b/.history/index_20231115220644.html new file mode 100644 index 0000000..946e1ac --- /dev/null +++ b/.history/index_20231115220644.html @@ -0,0 +1,608 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + TableLlama: Towards Open Large Generalist Models for Tables + + + + + + + + + + + + + + + + + + + + + + +
+
+
+
+
+ +
+ Icon +

TableLlama: Towards Open Large Generalist Models for Tables

+
+ +
+ + + 1Tianshu Zhang*, + + 2Xiang Yue, + + 1Yifei Li, + + 1Huan Sun + +
+ +
+ + 1The Ohio State University, + 2IN.AI + +
+ zhang.11535@osu.edu + , sun.397@osu.edu + +
+ + +
+
+ + + + + + + + + + + Code + + + + + + + + + + arXiv + + +
+
+
+
+
+
+
+ + + +
+
+
+
+
+

Abstract

+
+

+ Semi-structured tables are ubiquitous. There has been a variety of tasks that aim to automatically interpret, augment, and query tables. Current methods often require pretraining on tables or special model architecture design, are restricted to specific table types, or have simplifying assumptions about tables and tasks. This paper makes the first step towards developing open-source large language models (LLMs) as generalists for a diversity of table-based tasks. Towards that end, we construct TableInstruct, a new dataset with a variety of realistic tables and tasks, for instruction tuning and evaluating LLMs. We further develop the first open-source generalist model for tables, TableLlama, by fine-tuning Llama 2 (7B) with LongLoRA to address the long context challenge. We experiment under both in-domain setting and out-of-domain setting. On 7 out of 8 in-domain tasks, TableLlama achieves com- parable or better performance than the SOTA for each task, despite the latter often has task-specific design. On 6 out-of-domain datasets, it achieves 6-48 absolute point gains compared with the base model, showing that training on TableInstruct enhances the model’s generalizability. We will open-source our dataset and trained model to boost future work on developing open generalist models for tables. +

+
+
+
+
+
+
+ + + +
+
+
+
+
+
+ + An overview of TableInstruct and 🦙TableLlama +

+ Figure 1: An overview of TableInstruct and TableLlama. TableInstruct includes a wide variety of realistic tables and tasks with instructions. We make the first step towards developing open-source generalist models for tables with TableInstruct and TableLlama. +

+
+
+
+
+
+
+ + + +
+
+
+
+
+
+ + The hybrid instruction tuning of 🦣MAmmoTH +

+ Figure 2: Illustration of three exemplary tasks: (a) Column type annotation. This task is to annotate the selected column with the correct semantic types. (b) Row population. This task is to populate rows given table metadata and partial row entities. (c) Hierarchical table QA. For subfigures (a) and (b), we mark candidates with red color in the + "task instruction" part. The candidate set size can be hundreds to thousands in TableInstruct. +

+
+
+
+
+
+
+ + +
+
+
+
+
+

Our Dataset: TableInstruct

+
+ +
+
+
+
+
+
+ +
+
+
+
+
+

In-domain Evaluation:

+
+ +
+
+
+
+
+
+ + +
+
+
+
+
+

Out-of-domain Evaluation:

+
+ +
+
+
+
+
+
+ + + + +
+
+

Reference

+ Please kindly cite our paper if you use our code, data, models or results: +

+ 
@misc{zhang2023tablellama,
+  title={TableLlama: Towards Open Large Generalist Models for Tables}, 
+  author={Tianshu Zhang and Xiang Yue and Yifei Li and Huan Sun},
+  year={2023},
+  eprint={2311.09206},
+  archivePrefix={arXiv},
+  primaryClass={cs.CL}
+}
+
+
+
+ + + + + + + + + + + + + diff --git a/.history/index_20231115220722.html b/.history/index_20231115220722.html new file mode 100644 index 0000000..cd1574f --- /dev/null +++ b/.history/index_20231115220722.html @@ -0,0 +1,608 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + TableLlama: Towards Open Large Generalist Models for Tables + + + + + + + + + + + + + + + + + + + + + + +
+
+
+
+
+ +
+ Icon +

TableLlama: Towards Open Large Generalist Models for Tables

+
+ +
+ + + 1Tianshu Zhang*, + + 2Xiang Yue, + + 1Yifei Li, + + 1Huan Sun + +
+ +
+ + 1The Ohio State University, + 2IN.AI + +
+ zhang.11535@osu.edu + , sun.397@osu.edu + +
+ + +
+
+ + + + + + + + + + + Code + + + + + + + + + + arXiv + + +
+
+
+
+
+
+
+ + + +
+
+
+
+
+

Abstract

+
+

+ Semi-structured tables are ubiquitous. There has been a variety of tasks that aim to automatically interpret, augment, and query tables. Current methods often require pretraining on tables or special model architecture design, are restricted to specific table types, or have simplifying assumptions about tables and tasks. This paper makes the first step towards developing open-source large language models (LLMs) as generalists for a diversity of table-based tasks. Towards that end, we construct TableInstruct, a new dataset with a variety of realistic tables and tasks, for instruction tuning and evaluating LLMs. We further develop the first open-source generalist model for tables, TableLlama, by fine-tuning Llama 2 (7B) with LongLoRA to address the long context challenge. We experiment under both in-domain setting and out-of-domain setting. On 7 out of 8 in-domain tasks, TableLlama achieves com- parable or better performance than the SOTA for each task, despite the latter often has task-specific design. On 6 out-of-domain datasets, it achieves 6-48 absolute point gains compared with the base model, showing that training on TableInstruct enhances the model’s generalizability. We will open-source our dataset and trained model to boost future work on developing open generalist models for tables. +

+
+
+
+
+
+
+ + + +
+
+
+
+
+
+ + An overview of TableInstruct and 🦙TableLlama +

+ Figure 1: An overview of TableInstruct and TableLlama. TableInstruct includes a wide variety of realistic tables and tasks with instructions. We make the first step towards developing open-source generalist models for tables with TableInstruct and TableLlama. +

+
+
+
+
+
+
+ + + +
+
+
+
+
+
+ + The hybrid instruction tuning of 🦣MAmmoTH +

+ Figure 2: Illustration of three exemplary tasks: (a) Column type annotation. This task is to annotate the selected column with the correct semantic types. (b) Row population. This task is to populate rows given table metadata and partial row entities. (c) Hierarchical table QA. For subfigures (a) and (b), we mark candidates with red color in the + "task instruction" part. The candidate set size can be hundreds to thousands in TableInstruct. +

+
+
+
+
+
+
+ + +
+
+
+
+
+

Our Dataset: TableInstruct

+
+ +
+
+
+
+
+
+ +
+
+
+
+
+

In-domain Evaluation:

+
+ +
+
+
+
+
+
+ + +
+
+
+
+
+

Out-of-domain Evaluation:

+
+ +
+
+
+
+
+
+ + + + +
+
+

Reference

+ Please kindly cite our paper if you use our code, data, models or results: +

+ 
@misc{zhang2023tablellama,
+  title={TableLlama: Towards Open Large Generalist Models for Tables}, 
+  author={Tianshu Zhang and Xiang Yue and Yifei Li and Huan Sun},
+  year={2023},
+  eprint={2311.09206},
+  archivePrefix={arXiv},
+  primaryClass={cs.CL}
+}
+
+
+
+ + + + + + + + + + + + + diff --git a/.history/index_20231115220735.html b/.history/index_20231115220735.html new file mode 100644 index 0000000..0400394 --- /dev/null +++ b/.history/index_20231115220735.html @@ -0,0 +1,608 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + TableLlama: Towards Open Large Generalist Models for Tables + + + + + + + + + + + + + + + + + + + + + + +
+
+
+
+
+ +
+ Icon +

TableLlama: Towards Open Large Generalist Models for Tables

+
+ +
+ + + 1Tianshu Zhang*, + + 2Xiang Yue, + + 1Yifei Li, + + 1Huan Sun + +
+ +
+ + 1The Ohio State University, + 2IN.AI + +
+ zhang.11535@osu.edu + , sun.397@osu.edu + +
+ + +
+
+ + + + + + + + + + + Code + + + + + + + + + + arXiv + + +
+
+
+
+
+
+
+ + + +
+
+
+
+
+

Abstract

+
+

+ Semi-structured tables are ubiquitous. There has been a variety of tasks that aim to automatically interpret, augment, and query tables. Current methods often require pretraining on tables or special model architecture design, are restricted to specific table types, or have simplifying assumptions about tables and tasks. This paper makes the first step towards developing open-source large language models (LLMs) as generalists for a diversity of table-based tasks. Towards that end, we construct TableInstruct, a new dataset with a variety of realistic tables and tasks, for instruction tuning and evaluating LLMs. We further develop the first open-source generalist model for tables, TableLlama, by fine-tuning Llama 2 (7B) with LongLoRA to address the long context challenge. We experiment under both in-domain setting and out-of-domain setting. On 7 out of 8 in-domain tasks, TableLlama achieves com- parable or better performance than the SOTA for each task, despite the latter often has task-specific design. On 6 out-of-domain datasets, it achieves 6-48 absolute point gains compared with the base model, showing that training on TableInstruct enhances the model’s generalizability. We will open-source our dataset and trained model to boost future work on developing open generalist models for tables. +

+
+
+
+
+
+
+ + + +
+
+
+
+
+
+ + An overview of TableInstruct and 🦙TableLlama +

+ Figure 1: An overview of TableInstruct and TableLlama. TableInstruct includes a wide variety of realistic tables and tasks with instructions. We make the first step towards developing open-source generalist models for tables with TableInstruct and TableLlama. +

+
+
+
+
+
+
+ + + +
+
+
+
+
+
+ + The hybrid instruction tuning of 🦣MAmmoTH +

+ Figure 2: Illustration of three exemplary tasks: (a) Column type annotation. This task is to annotate the selected column with the correct semantic types. (b) Row population. This task is to populate rows given table metadata and partial row entities. (c) Hierarchical table QA. For subfigures (a) and (b), we mark candidates with red color in the + "task instruction" part. The candidate set size can be hundreds to thousands in TableInstruct. +

+
+
+
+
+
+
+ + +
+
+
+
+
+

Our Dataset: TableInstruct

+
+ +
+
+
+
+
+
+ +
+
+
+
+
+

In-domain Evaluation:

+
+ +
+
+
+
+
+
+ + +
+
+
+
+
+

Out-of-domain Evaluation:

+
+ +
+
+
+
+
+
+ + + + +
+
+

Reference

+ Please kindly cite our paper if you use our code, data, models or results: +

+ 
@misc{zhang2023tablellama,
+  title={TableLlama: Towards Open Large Generalist Models for Tables}, 
+  author={Tianshu Zhang and Xiang Yue and Yifei Li and Huan Sun},
+  year={2023},
+  eprint={2311.09206},
+  archivePrefix={arXiv},
+  primaryClass={cs.CL}
+}
+
+
+
+ + + + + + + + + + + + + diff --git a/index.html b/index.html index 2ba7a05..0400394 100644 --- a/index.html +++ b/index.html @@ -4,12 +4,12 @@ - - - - + + + + - + @@ -123,26 +123,8 @@

TableLlama: Towards Open Large Generali
- - - - - - Paper - - - - - - - - Supplementary - -