Asking TableLlama to answer out-of-domain questions

[giangnguyen2412]

Hello again,

I am running inference with TableLlama on FetaQA, for the generated output, i have this example entry:

  {
    "idx": 6,
    "instruction": "This is a free-form table question answering task. The goal for this task is to answer the given question based on the given table and the highlighted cells.",
    "input_seg": "[TLE] The Wikipedia page title of this table is Roscoe Parrish. The Wikipedia section title of this table is Personal bests. [TAB] | Event | Time (seconds) | Venue | Date | [SEP] | 55 meters | 6.38 | Gainesville, Florida | January 18, 2003 | [SEP] | 60 meters | 6.89 | Syracuse, New York | February 16, 2002 | [SEP] | 100 meters | 10.65 | Coral Gables, Florida | April 12, 2003 | [SEP] | 200 meters | 21.13 | Storrs, Connecticut | May 4, 2003 |",
    "question": "The highlighted cells of the table are: [HIGHLIGHTED_BEGIN] [55 meters], [6.38], [Gainesville, Florida], [January 18, 2003] [HIGHLIGHTED_END] What is the personal best of Roscoe Parrish in 55 meters, and where and when did he make it?'"
    "output": "Roscoe Parrish competed in the 55 meters, posting a personal best time of 6.38 seconds in Gainesville, Florida on\u00a0January 18, 2003\u00a0",
    "predict": "Roscoe Parrish made a personal best of 6.38 seconds in the 55 m event at Gainesville, Florida on January 18, 2003.</s>"
  },

Could I append another question to the "question" field to ask TableLlama to provide more information beyond the the answer for the question-answering task.

For example, the "question" field could be extended as:

"The highlighted cells of the table are: [HIGHLIGHTED_BEGIN] [55 meters], [6.38], [Gainesville, Florida], [January 18, 2003] [HIGHLIGHTED_END] What is the personal best of Roscoe Parrish in 55 meters, and where and when did he make it? . **Please include the highlighted information in your response.**"

However, I found the TableLlama still answers the same and seems to ignore the added question. Is this behavior expected and what should I do to make Llama behaves like other LLMs (i.e. considering all aspects in the user queries)

Thanks for the great and prompt support!

[giangnguyen2412]

even when the question field is empty

"question": "",

, the model still generates the same things