-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy path04_initial_summarization.py
56 lines (45 loc) · 15.6 KB
/
04_initial_summarization.py
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
"""
Attempts to summarize.
"""
import copy, json, logging, pprint
from llama_cpp import Llama
logging.basicConfig(
level=logging.DEBUG,
format='[%(asctime)s] %(levelname)s [%(module)s-%(funcName)s()::%(lineno)d] %(message)s',
datefmt='%d/%b/%Y %H:%M:%S' )
log = logging.getLogger( '__name__' )
## load model -------------------------------------------------------
log.debug( 'loading model' )
llm = Llama(
model_path='../models/ggml-vicuna-13b-4bit-rev1.bin',
n_ctx=4000 )
log.debug( 'model loaded' )
## run model --------------------------------------------------------
log.debug( 'running model' )
## testing 1,500 (1,498) words
obama_speech = '''
Thank you so much. Tonight, more than 200 years after a former colony won the right to determine its own destiny, the task of perfecting our union moves forward. It moves forward because of you. It moves forward because you reaffirmed the spirit that has triumphed over war and depression, the spirit that has lifted this country from the depths of despair to the great heights of hope, the belief that while each of us will pursue our own individual dreams, we are an American family and we rise or fall together as one nation and as one people. Tonight, in this election, you, the American people, reminded us that while our road has been hard, while our journey has been long, we have picked ourselves up, we have fought our way back, and we know in our hearts that for the United States of America the best is yet to come. I want to thank every American who participated in this election, whether you voted for the very first time or waited in line for a very long time. By the way, we have to fix that. Whether you pounded the pavement or picked up the phone, whether you held an Obama sign or a Romney sign, you made your voice heard and you made a difference. I just spoke with Gov. Romney and I congratulated him and Paul Ryan on a hard-fought campaign. We may have battled fiercely, but it's only because we love this country deeply and we care so strongly about its future. From George to Lenore to their son Mitt, the Romney family has chosen to give back to America through public service and that is the legacy that we honor and applaud tonight. In the weeks ahead, I also look forward to sitting down with Gov. Romney to talk about where we can work together to move this country forward. I want to thank my friend and partner of the last four years, America's happy warrior, the best vice president anybody could ever hope for, Joe Biden. And I wouldn't be the man I am today without the woman who agreed to marry me 20 years ago. Let me say this publicly: Michelle, I have never loved you more. I have never been prouder to watch the rest of America fall in love with you, too, as our nation's first lady. Sasha and Malia, before our very eyes you're growing up to become two strong, smart beautiful young women, just like your mom. And I'm so proud of you guys. But I will say that for now one dog's probably enough. To the best campaign team and volunteers in the history of politics. The best. The best ever. Some of you were new this time around, and some of you have been at my side since the very beginning. But all of you are family. No matter what you do or where you go from here, you will carry the memory of the history we made together and you will have the lifelong appreciation of a grateful president. Thank you for believing all the way, through every hill, through every valley. You lifted me up the whole way and I will always be grateful for everything that you've done and all the incredible work that you put in. I know that political campaigns can sometimes seem small, even silly. And that provides plenty of fodder for the cynics that tell us that politics is nothing more than a contest of egos or the domain of special interests. But if you ever get the chance to talk to folks who turned out at our rallies and crowded along a rope line in a high school gym, or saw folks working late in a campaign office in some tiny county far away from home, you'll discover something else. You'll hear the determination in the voice of a young field organizer who's working his way through college and wants to make sure every child has that same opportunity. You'll hear the pride in the voice of a volunteer who's going door to door because her brother was finally hired when the local auto plant added another shift. You'll hear the deep patriotism in the voice of a military spouse who's working the phones late at night to make sure that no one who fights for this country ever has to fight for a job or a roof over their head when they come home. That's why we do this. That's what politics can be. That's why elections matter. It's not small, it's big. It's important. Democracy in a nation of 300 million can be noisy and messy and complicated. We have our own opinions. Each of us has deeply held beliefs. And when we go through tough times, when we make big decisions as a country, it necessarily stirs passions, stirs up controversy. That won't change after tonight, and it shouldn't. These arguments we have are a mark of our liberty. We can never forget that as we speak people in distant nations are risking their lives right now just for a chance to argue about the issues that matter, the chance to cast their ballots like we did today. But despite all our differences, most of us share certain hopes for America's future. We want our kids to grow up in a country where they have access to the best schools and the best teachers. A country that lives up to its legacy as the global leader in technology and discovery and innovation, with all the good jobs and new businesses that follow. We want our children to live in an America that isn't burdened by debt, that isn't weakened by inequality, that isn't threatened by the destructive power of a warming planet. We want to pass on a country that's safe and respected and admired around the world, a nation that is defended by the strongest military on earth and the best troops this -- this world has ever known. But also a country that moves with confidence beyond this time of war, to shape a peace that is built on the promise of freedom and dignity for every human being. We believe in a generous America, in a compassionate America, in a tolerant America, open to the dreams of an immigrant's daughter who studies in our schools and pledges to our flag. To the young boy on the south side of Chicago who sees a life beyond the nearest street corner. To the furniture worker's child in North Carolina who wants to become a doctor or a scientist, an engineer or an entrepreneur, a diplomat or even a president -- that's the future we hope for. That's the vision we share. That's where we need to go -- forward. That's where we need to go. Now, we will disagree, sometimes fiercely, about how to get there. As it has for more than two centuries, progress will come in fits and starts. It's not always a straight line. It's not always a smooth path. By itself, the recognition that we have common hopes and dreams won't end all the gridlock or solve all our problems or substitute for the painstaking work of building consensus and making the difficult compromises needed to move this country forward. But that common bond is where we must begin. Our economy is recovering. A decade of war is ending. A long campaign is now over. And whether I earned your vote or not, I have listened to you, I have learned from you, and you've made me a better president. And with your stories and your struggles, I return to the White House more determined and more inspired than ever about the work there is to do and the future that lies ahead. Tonight you voted for action, not politics as usual. You elected us to focus on your jobs, not ours. And in the coming weeks and months, I am looking forward to reaching out and working with leaders of both parties to meet the challenges we can only solve together. Reducing our deficit. Reforming our tax code. Fixing our immigration system. Freeing ourselves from foreign oil. We've got more work to do. But that doesn't mean your work is done. The role of citizen in our democracy does not end with your vote. America's never been about what can be done for us. It's about what can be done by us together through the hard and frustrating, but necessary work of self-government. That's the principle we were founded on. This country has more wealth than any nation, but that's not what makes us rich. We have the most powerful military in history, but that's not what makes us strong. Our university, our culture are all the envy of the world, but that's not what keeps the world coming to our shores. What makes America exceptional are the bonds that hold together the most diverse nation on earth.
'''
## testing 1,005 words -- ok summary, but only seems to summarize the beginning of the talk, and doesn't complete the final sentence.
# obama_speech = '''
# Thank you so much. Tonight, more than 200 years after a former colony won the right to determine its own destiny, the task of perfecting our union moves forward. It moves forward because of you. It moves forward because you reaffirmed the spirit that has triumphed over war and depression, the spirit that has lifted this country from the depths of despair to the great heights of hope, the belief that while each of us will pursue our own individual dreams, we are an American family and we rise or fall together as one nation and as one people. Tonight, in this election, you, the American people, reminded us that while our road has been hard, while our journey has been long, we have picked ourselves up, we have fought our way back, and we know in our hearts that for the United States of America the best is yet to come. I want to thank every American who participated in this election, whether you voted for the very first time or waited in line for a very long time. By the way, we have to fix that. Whether you pounded the pavement or picked up the phone, whether you held an Obama sign or a Romney sign, you made your voice heard and you made a difference. I just spoke with Gov. Romney and I congratulated him and Paul Ryan on a hard-fought campaign. We may have battled fiercely, but it's only because we love this country deeply and we care so strongly about its future. From George to Lenore to their son Mitt, the Romney family has chosen to give back to America through public service and that is the legacy that we honor and applaud tonight. In the weeks ahead, I also look forward to sitting down with Gov. Romney to talk about where we can work together to move this country forward. I want to thank my friend and partner of the last four years, America's happy warrior, the best vice president anybody could ever hope for, Joe Biden. And I wouldn't be the man I am today without the woman who agreed to marry me 20 years ago. Let me say this publicly: Michelle, I have never loved you more. I have never been prouder to watch the rest of America fall in love with you, too, as our nation's first lady. Sasha and Malia, before our very eyes you're growing up to become two strong, smart beautiful young women, just like your mom. And I'm so proud of you guys. But I will say that for now one dog's probably enough. To the best campaign team and volunteers in the history of politics. The best. The best ever. Some of you were new this time around, and some of you have been at my side since the very beginning. But all of you are family. No matter what you do or where you go from here, you will carry the memory of the history we made together and you will have the lifelong appreciation of a grateful president. Thank you for believing all the way, through every hill, through every valley. You lifted me up the whole way and I will always be grateful for everything that you've done and all the incredible work that you put in. I know that political campaigns can sometimes seem small, even silly. And that provides plenty of fodder for the cynics that tell us that politics is nothing more than a contest of egos or the domain of special interests. But if you ever get the chance to talk to folks who turned out at our rallies and crowded along a rope line in a high school gym, or saw folks working late in a campaign office in some tiny county far away from home, you'll discover something else. You'll hear the determination in the voice of a young field organizer who's working his way through college and wants to make sure every child has that same opportunity. You'll hear the pride in the voice of a volunteer who's going door to door because her brother was finally hired when the local auto plant added another shift. You'll hear the deep patriotism in the voice of a military spouse who's working the phones late at night to make sure that no one who fights for this country ever has to fight for a job or a roof over their head when they come home. That's why we do this. That's what politics can be. That's why elections matter. It's not small, it's big. It's important. Democracy in a nation of 300 million can be noisy and messy and complicated. We have our own opinions. Each of us has deeply held beliefs. And when we go through tough times, when we make big decisions as a country, it necessarily stirs passions, stirs up controversy. That won't change after tonight, and it shouldn't. These arguments we have are a mark of our liberty. We can never forget that as we speak people in distant nations are risking their lives right now just for a chance to argue about the issues that matter, the chance to cast their ballots like we did today. But despite all our differences, most of us share certain hopes for America's future. We want our kids to grow up in a country where they have access to the best schools and the best teachers. A country that lives up to its legacy as the global leader in technology and discovery and innovation, with all the good jobs and new businesses that follow. We want our children to live in an America that isn't burdened by debt, that isn't weakened by inequality, that isn't threatened by the destructive power of a warming planet. We want to pass on a country that's safe and respected and admired around the world, a nation that is defended by the strongest military on earth and the best troops this -- this world has ever known.
# '''
## testing 188 words works well
# obama_speech = '''
# Thank you so much. Tonight, more than 200 years after a former colony won the right to determine its own destiny, the task of perfecting our union moves forward. It moves forward because of you. It moves forward because you reaffirmed the spirit that has triumphed over war and depression, the spirit that has lifted this country from the depths of despair to the great heights of hope, the belief that while each of us will pursue our own individual dreams, we are an American family and we rise or fall together as one nation and as one people. Tonight, in this election, you, the American people, reminded us that while our road has been hard, while our journey has been long, we have picked ourselves up, we have fought our way back, and we know in our hearts that for the United States of America the best is yet to come. I want to thank every American who participated in this election, whether you voted for the very first time or waited in line for a very long time. By the way, we have to fix that.
# '''
message = f'Summarize the following text (75-word-maximum) using a neutral tone, describing main themes and topics. The text: {obama_speech}'
log.debug( f'message, ``{message}``' )
messages = [ {'role': 'user', 'content': message} ]
output = llm.create_chat_completion(
messages,
temperature=0.2,
top_p=0.95,
top_k=40,
stream=False,
stop=[],
max_tokens=100,
repeat_penalty=1.1
)
## show output ------------------------------------------------------
log.debug( f'output, ``{pprint.pformat(output)}``' )