public class StringTokenizer extends Object implements ListIterator<String>, Cloneable
This class can split a String into many smaller strings. It aims to do a similar job to
StringTokenizer
, however it offers much more control and flexibility including
implementing the ListIterator
interface. By default, it is set up like StringTokenizer
.
The input String is split into a number of tokens. Each token is separated from the next String by a delimiter. One or more delimiter characters must be specified.
Each token may be surrounded by quotes. The quote matcher specifies the quote character(s). A quote may be escaped within a quoted section by duplicating itself.
Between each token and the delimiter are potentially characters that need trimming. The trimmer matcher specifies these characters. One usage might be to trim whitespace characters.
At any point outside the quotes there might potentially be invalid characters. The ignored matcher specifies these characters to be removed. One usage might be to remove new line characters.
Empty tokens may be removed or returned as null.
"a,b,c" - Three tokens "a","b","c" (comma delimiter) " a, b , c " - Three tokens "a","b","c" (default CSV processing trims whitespace) "a, ", b ,", c" - Three tokens "a, " , " b ", ", c" (quoted text untouched)
Property | Type | Default |
---|---|---|
delim | CharSetMatcher | { \t\n\r\f} |
quote | NoneMatcher | {} |
ignore | NoneMatcher | {} |
emptyTokenAsNull | boolean | false |
ignoreEmptyTokens | boolean | true |
Constructor and Description |
---|
StringTokenizer()
Constructs a tokenizer splitting on space, tab, newline and form feed as per StringTokenizer, but with no text to
tokenize.
|
StringTokenizer(char[] input)
Constructs a tokenizer splitting on space, tab, newline and form feed as per StringTokenizer.
|
StringTokenizer(char[] input,
char delim)
Constructs a tokenizer splitting on the specified character.
|
StringTokenizer(char[] input,
char delim,
char quote)
Constructs a tokenizer splitting on the specified delimiter character and handling quotes using the specified
quote character.
|
StringTokenizer(char[] input,
String delim)
Constructs a tokenizer splitting on the specified string.
|
StringTokenizer(char[] input,
StringMatcher delim)
Constructs a tokenizer splitting using the specified delimiter matcher.
|
StringTokenizer(char[] input,
StringMatcher delim,
StringMatcher quote)
Constructs a tokenizer splitting using the specified delimiter matcher and handling quotes using the specified
quote matcher.
|
StringTokenizer(String input)
Constructs a tokenizer splitting on space, tab, newline and form feed as per StringTokenizer.
|
StringTokenizer(String input,
char delim)
Constructs a tokenizer splitting on the specified delimiter character.
|
StringTokenizer(String input,
char delim,
char quote)
Constructs a tokenizer splitting on the specified delimiter character and handling quotes using the specified
quote character.
|
StringTokenizer(String input,
String delim)
Constructs a tokenizer splitting on the specified delimiter string.
|
StringTokenizer(String input,
StringMatcher delim)
Constructs a tokenizer splitting using the specified delimiter matcher.
|
StringTokenizer(String input,
StringMatcher delim,
StringMatcher quote)
Constructs a tokenizer splitting using the specified delimiter matcher and handling quotes using the specified
quote matcher.
|
Modifier and Type | Method and Description |
---|---|
void |
add(String obj)
Unsupported ListIterator operation.
|
Object |
clone()
Creates a new instance of this Tokenizer.
|
String |
getContent()
Gets the String content that the tokenizer is parsing.
|
static StringTokenizer |
getCSVInstance()
Gets a new tokenizer instance which parses Comma Separated Value strings initializing it with the given input.
|
static StringTokenizer |
getCSVInstance(char[] input)
Gets a new tokenizer instance which parses Comma Separated Value strings initializing it with the given input.
|
static StringTokenizer |
getCSVInstance(String input)
Gets a new tokenizer instance which parses Comma Separated Value strings initializing it with the given input.
|
StringMatcher |
getDelimiterMatcher()
Gets the field delimiter matcher.
|
StringMatcher |
getIgnoredMatcher()
Gets the ignored character matcher.
|
StringMatcher |
getQuoteMatcher()
Gets the quote matcher currently in use.
|
String[] |
getTokenArray()
Gets a copy of the full token list as an independent modifiable array.
|
List<String> |
getTokenList()
Gets a copy of the full token list as an independent modifiable list.
|
StringMatcher |
getTrimmerMatcher()
Gets the trimmer character matcher.
|
static StringTokenizer |
getTSVInstance()
Gets a new tokenizer instance which parses Tab Separated Value strings.
|
static StringTokenizer |
getTSVInstance(char[] input)
Gets a new tokenizer instance which parses Tab Separated Value strings.
|
static StringTokenizer |
getTSVInstance(String input)
Gets a new tokenizer instance which parses Tab Separated Value strings.
|
boolean |
hasNext()
Checks whether there are any more tokens.
|
boolean |
hasPrevious()
Checks whether there are any previous tokens that can be iterated to.
|
boolean |
isEmptyTokenAsNull()
Gets whether the tokenizer currently returns empty tokens as null.
|
boolean |
isIgnoreEmptyTokens()
Gets whether the tokenizer currently ignores empty tokens.
|
String |
next()
Gets the next token.
|
int |
nextIndex()
Gets the index of the next token to return.
|
String |
nextToken()
Gets the next token from the String.
|
String |
previous()
Gets the token previous to the last returned token.
|
int |
previousIndex()
Gets the index of the previous token.
|
String |
previousToken()
Gets the previous token from the String.
|
void |
remove()
Unsupported ListIterator operation.
|
StringTokenizer |
reset()
Resets this tokenizer, forgetting all parsing and iteration already completed.
|
StringTokenizer |
reset(char[] input)
Reset this tokenizer, giving it a new input string to parse.
|
StringTokenizer |
reset(String input)
Reset this tokenizer, giving it a new input string to parse.
|
void |
set(String obj)
Unsupported ListIterator operation.
|
StringTokenizer |
setDelimiterChar(char delim)
Sets the field delimiter character.
|
StringTokenizer |
setDelimiterMatcher(StringMatcher delim)
Sets the field delimiter matcher.
|
StringTokenizer |
setDelimiterString(String delim)
Sets the field delimiter string.
|
StringTokenizer |
setEmptyTokenAsNull(boolean emptyAsNull)
Sets whether the tokenizer should return empty tokens as null.
|
StringTokenizer |
setIgnoredChar(char ignored)
Set the character to ignore.
|
StringTokenizer |
setIgnoredMatcher(StringMatcher ignored)
Set the matcher for characters to ignore.
|
StringTokenizer |
setIgnoreEmptyTokens(boolean ignoreEmptyTokens)
Sets whether the tokenizer should ignore and not return empty tokens.
|
StringTokenizer |
setQuoteChar(char quote)
Sets the quote character to use.
|
StringTokenizer |
setQuoteMatcher(StringMatcher quote)
Set the quote matcher to use.
|
StringTokenizer |
setTrimmerMatcher(StringMatcher trimmer)
Sets the matcher for characters to trim.
|
int |
size()
Gets the number of tokens found in the String.
|
protected List<String> |
tokenize(char[] srcChars,
int offset,
int count)
Internal method to performs the tokenization.
|
String |
toString()
Gets the String content that the tokenizer is parsing.
|
equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
forEachRemaining
public StringTokenizer()
This constructor is normally used with reset(String)
.
public StringTokenizer(String input)
input
- the string which is to be parsedpublic StringTokenizer(String input, char delim)
input
- the string which is to be parseddelim
- the field delimiter characterpublic StringTokenizer(String input, String delim)
input
- the string which is to be parseddelim
- the field delimiter stringpublic StringTokenizer(String input, StringMatcher delim)
input
- the string which is to be parseddelim
- the field delimiter matcherpublic StringTokenizer(String input, char delim, char quote)
input
- the string which is to be parseddelim
- the field delimiter characterquote
- the field quoted string characterpublic StringTokenizer(String input, StringMatcher delim, StringMatcher quote)
input
- the string which is to be parseddelim
- the field delimiter matcherquote
- the field quoted string matcherpublic StringTokenizer(char[] input)
input
- the string which is to be parsed, not clonedpublic StringTokenizer(char[] input, char delim)
input
- the string which is to be parsed, not cloneddelim
- the field delimiter characterpublic StringTokenizer(char[] input, String delim)
input
- the string which is to be parsed, not cloneddelim
- the field delimiter stringpublic StringTokenizer(char[] input, StringMatcher delim)
input
- the string which is to be parsed, not cloneddelim
- the field delimiter matcherpublic StringTokenizer(char[] input, char delim, char quote)
input
- the string which is to be parsed, not cloneddelim
- the field delimiter characterquote
- the field quoted string characterpublic StringTokenizer(char[] input, StringMatcher delim, StringMatcher quote)
input
- the string which is to be parsed, not cloneddelim
- the field delimiter characterquote
- the field quoted string characterpublic static StringTokenizer getCSVInstance()
You must call a "reset" method to set the string which you want to parse.
public static StringTokenizer getCSVInstance(String input)
input
- the text to parsepublic static StringTokenizer getCSVInstance(char[] input)
input
- the text to parsepublic static StringTokenizer getTSVInstance()
You must call a "reset" method to set the string which you want to parse.
public static StringTokenizer getTSVInstance(String input)
input
- the string to parsepublic static StringTokenizer getTSVInstance(char[] input)
input
- the string to parsepublic int size()
public String nextToken()
next()
except it returns null rather than throwing
NoSuchElementException
when no tokens remain.public String previousToken()
public String[] getTokenArray()
public List<String> getTokenList()
public StringTokenizer reset()
This method allows the same tokenizer to be reused for the same String.
public StringTokenizer reset(String input)
input
- the new string to tokenize, null sets no text to parsepublic StringTokenizer reset(char[] input)
input
- the new character array to tokenize, not cloned, null sets no text to parsepublic boolean hasNext()
public String next()
next
in interface Iterator<String>
next
in interface ListIterator<String>
NoSuchElementException
- if there are no more elementspublic int nextIndex()
nextIndex
in interface ListIterator<String>
public boolean hasPrevious()
hasPrevious
in interface ListIterator<String>
public String previous()
previous
in interface ListIterator<String>
public int previousIndex()
previousIndex
in interface ListIterator<String>
public void remove()
remove
in interface Iterator<String>
remove
in interface ListIterator<String>
UnsupportedOperationException
- alwayspublic void set(String obj)
set
in interface ListIterator<String>
obj
- this parameter ignored.UnsupportedOperationException
- alwayspublic void add(String obj)
add
in interface ListIterator<String>
obj
- this parameter ignored.UnsupportedOperationException
- alwaysprotected List<String> tokenize(char[] srcChars, int offset, int count)
Most users of this class do not need to call this method. This method will be called automatically by other (public) methods when required.
This method exists to allow subclasses to add code before or after the tokenization. For example, a subclass could alter the character array, offset or count to be parsed, or call the tokenizer multiple times on multiple strings. It is also be possible to filter the results.
StrTokenizer
will always pass a zero offset and a count equal to the length of the array to this
method, however a subclass may pass other values, or even an entirely different array.
srcChars
- the character array being tokenized, may be nulloffset
- the start position within the character array, must be validcount
- the number of characters to tokenize, must be validpublic StringMatcher getDelimiterMatcher()
public StringTokenizer setDelimiterMatcher(StringMatcher delim)
The delimiter is used to separate one token from another.
delim
- the delimiter matcher to usepublic StringTokenizer setDelimiterChar(char delim)
delim
- the delimiter character to usepublic StringTokenizer setDelimiterString(String delim)
delim
- the delimiter string to usepublic StringMatcher getQuoteMatcher()
The quote character is used to wrap data between the tokens. This enables delimiters to be entered as data. The default value is '"' (double quote).
public StringTokenizer setQuoteMatcher(StringMatcher quote)
The quote character is used to wrap data between the tokens. This enables delimiters to be entered as data.
quote
- the quote matcher to use, null ignoredpublic StringTokenizer setQuoteChar(char quote)
The quote character is used to wrap data between the tokens. This enables delimiters to be entered as data.
quote
- the quote character to usepublic StringMatcher getIgnoredMatcher()
These characters are ignored when parsing the String, unless they are within a quoted region. The default value is not to ignore anything.
public StringTokenizer setIgnoredMatcher(StringMatcher ignored)
These characters are ignored when parsing the String, unless they are within a quoted region.
ignored
- the ignored matcher to use, null ignoredpublic StringTokenizer setIgnoredChar(char ignored)
This character is ignored when parsing the String, unless it is within a quoted region.
ignored
- the ignored character to usepublic StringMatcher getTrimmerMatcher()
These characters are trimmed off on each side of the delimiter until the token or quote is found. The default value is not to trim anything.
public StringTokenizer setTrimmerMatcher(StringMatcher trimmer)
These characters are trimmed off on each side of the delimiter until the token or quote is found.
trimmer
- the trimmer matcher to use, null ignoredpublic boolean isEmptyTokenAsNull()
public StringTokenizer setEmptyTokenAsNull(boolean emptyAsNull)
emptyAsNull
- whether empty tokens are returned as nullpublic boolean isIgnoreEmptyTokens()
public StringTokenizer setIgnoreEmptyTokens(boolean ignoreEmptyTokens)
ignoreEmptyTokens
- whether empty tokens are not returnedpublic String getContent()
public Object clone()
CloneNotSupportedException
is caught, return null
.Copyright © 2014–2020 The Apache Software Foundation. All rights reserved.