Abstractive Summarization Using Pytorch
Created on 2023-03-21T08:41:01-05:00
Ensure Hugging Face's transformers are installed:
pip install transformers
Python stanza:
from transformers import pipeline import os os.environ["CUDA_VISIBLE_DEVICES"] = "0" # optional summarizer = pipeline("summarization") # BART model summarizer = pipeline("summarization", model="t5-base", tokenizer="t5-base", framework="tf") # t5-base model text="""...""" summary_text = summarizer(text, max_length=100, min_length=5, do_sample=False)[0]['summary_text'] print(summary_text)
T5 is trained on Common Crawl (so, Reddit and random internet shit) while BART is trained on CNN and Daily Mail.