Node parsers are a simple abstraction that take a list of documents, and chunk them into
Node objects, such that each node is a specific size. When a document is broken into nodes, all of it’s attributes are inherited to the children nodes (i.e.
metadata, text and metadata templates, etc.). You can read more about
Document properties here.
A node parser can configure the chunk size (in tokens) as well as any overlap between chunked nodes. The chunking is done by using a
TokenTextSplitter, which default to a chunk size of 1024 and a default chunk overlap of 20 tokens.