The Artima Developer Community
Sponsored Link

Java Answers Forum
Problem in Parsing Strings using StringTokenizer

10 replies on 1 page. Most recent reply: Jun 13, 2003 6:38 PM by Lee Pederson

Welcome Guest
  Sign In

Go back to the topic listing  Back to Topic List Click to reply to this topic  Reply to this Topic Click to search messages in this forum  Search Forum Click for a threaded view of the topic  Threaded View   
Previous Topic   Next Topic
Flat View: This topic has 10 replies on 1 page
Venkatesan

Posts: 9
Nickname: venkki
Registered: Jun, 2003

Problem in Parsing Strings using StringTokenizer Posted: Jun 9, 2003 3:16 AM
Reply to this message Reply
Advertisement
My application needs to parse a String with a double delimeter "::" into String arrays.
The code I am using is as follows.
String strTemp ="999::adm::888::mgr::security::flag";
StringTokenizer strToken = new StringTokenizer(strTemp,"::");
String tempArray[] = new String[7];
int index =0;
while (strToken.hasMoreTokens())
{
System.out.println("tempArray "+index+" :: "+(String)strToken.nextToken());
index++;
}
This format of String strTemp is working fine and I am getting six String arrays.
But when any value inbetween the double delimeter is missing,say example the data "888" is missing ,then
strTemp ="999::adm::::mgr::security::flag";
My requirement is , now also I should get the same no of arrays of Strings , inwhich array[2] should be null.
But I am getting only 5 arrays of Strings where I am seeing all the array indexs after the missed String is shifted.
StringTokenizer approach is not working at all .
I knew , we can solve this issue by doing IndexOf method of String Class..But I feel, this approach is a very basic one which consumes more no of coding lines..
Friends, anybody guide me with an optimised approach?


Adam Duffy

Posts: 168
Nickname: adamduffy
Registered: Feb, 2003

Re: Problem in Parsing Strings using StringTokenizer Posted: Jun 9, 2003 7:22 AM
Reply to this message Reply
Try this code.

String strTemp ="999::adm::::mgr::security::flag";
String token[] = strTemp.split( "::" );
for( int i = 0; i < token.length; i++ )
{
  System.out.println( "tempArray "+i+" :: " + token[i] );
}


The method split is useful in this situation and should return the array of strings in the manner that you desire with array[2] equal to null.

Adam

Venkatesan

Posts: 9
Nickname: venkki
Registered: Jun, 2003

Re: Problem in Parsing Strings using StringTokenizer Posted: Jun 9, 2003 11:31 PM
Reply to this message Reply
Hi...
Thanks to ur reply.
But the code u given is throwing compilation error since there is no split method in String Class of Java. I am using jdk1.3.

Paris Deligiannakis

Posts: 2
Nickname: customiser
Registered: May, 2003

Re: Problem in Parsing Strings using StringTokenizer Posted: Jun 10, 2003 2:05 AM
Reply to this message Reply
split was introduced in version 1.4

otherwise, you might want to try
StringTokenizer strToken = new StringTokenize(strTemp, "::", true);


This will also return the delimiters, so you'll get two consecutive tokens equal to your delimiters when a value is missing.

(Not 100% sure though, I can't try it out from here)

John Channing

Posts: 17
Nickname: drc
Registered: Jun, 2003

Re: Problem in Parsing Strings using StringTokenizer Posted: Jun 10, 2003 2:10 AM
Reply to this message Reply
It's easy enough to see the implementation of the StringTokenizer class, especially if you have an IDE like JBuilder. Just click on StringTokenizer somewhere in you code and press Ctrl-Enter. This quickly shows that the StringTokenizer approach is never going to work in the case where there is nothing between the delimiters. This is because the code loops until it finds something that isn't a delimiter character. A neat approach, if you have the programming skills to do it, would be to do a copy-paste-modify job to create you own version of the StringTokenizer class and save it in your own package.
John

Venkatesan

Posts: 9
Nickname: venkki
Registered: Jun, 2003

Re: Problem in Parsing Strings using StringTokenizer Posted: Jun 10, 2003 3:08 AM
Reply to this message Reply
Where can I find the Source Code for StringTokenizer Class?
Can u send it to me??

Singh M.

Posts: 154
Nickname: ms
Registered: Mar, 2002

Re: Problem in Parsing Strings using StringTokenizer Posted: Jun 10, 2003 3:25 AM
Reply to this message Reply
public final class JTexStringTokenizer{
    private static final String DEFAULT_DELIMITER = " ";
    private String _data;
    private String _delimiters;
    //private char [] charArray;
    
    private int currentElementPosition;
    private int currentTokenPosition;
    private int dataLength;
    
    public JTexStringTokenizer(String data){
        _data = data;
        dataLength = data.length();
        _delimiters = DEFAULT_DELIMITER;
    }
    
    public JTexStringTokenizer(String data, String delimiters){
        _data = data;
        dataLength = data.length();
        _delimiters = delimiters;
    }
    
    public final boolean hasMoreElements(){
        for (int i = currentTokenPosition; i < dataLength; i++){
            int dataToCheck = (int)(_data.charAt(i));
            
            if (_delimiters.indexOf(dataToCheck)== -1){
                currentElementPosition = i;
                return true;
            }
        }
        
        return false;
    }
    
    public final String nextElement(){
        for (int i = currentElementPosition; i < dataLength; i++){
            int dataToCheck = (int)(_data.charAt(i));
            
            if (_delimiters.indexOf(dataToCheck)!= -1){
                currentTokenPosition = i;
                break;
            }
        }
        
        if (currentTokenPosition < currentElementPosition){
            currentTokenPosition = dataLength;
        }
        
        return _data.substring(currentElementPosition, currentTokenPosition);
    }
    
    public final int getElementOffset(){
        return currentElementPosition;
    }
    
    public final int getTokenOffset(){
        return currentTokenPosition;
    }
    
    public final String getToken(){
        return String.valueOf(_data.charAt(currentTokenPosition));
    }
    
    public void reset(){
        currentElementPosition = 0;
        currentTokenPosition = 0;
    }
    
    public static void main(String [] args){
        JTexStringTokenizer tokenizer = new JTexStringTokenizer("999::adm::::mgr::security::flag", "::");
        
        while (tokenizer.hasMoreElements()){
            System.err.print(tokenizer.nextElement());
            System.err.println(" "+tokenizer.getElementOffset()+" "+tokenizer.getTokenOffset());
        }
    }
}


This is the code for a string tokenizer that i wrote for my java editor. Hope this helps.

Venkatesan

Posts: 9
Nickname: venkki
Registered: Jun, 2003

Re: Problem in Parsing Strings using StringTokenizer Posted: Jun 10, 2003 3:51 AM
Reply to this message Reply
Mr Singh M.,
The source code you given also is behaving like ordinary StringTokenizer as for as my issue is concerned. Can u help me where do I modify ur source code sothat I will get "null" value for the missed data inbetween 2 sucessive delimeters??

Ed Smiley

Posts: 4
Nickname: hce
Registered: Jun, 2003

Re: Problem in Parsing Strings using StringTokenizer Posted: Jun 10, 2003 2:28 PM
Reply to this message Reply
Actually another method occured to me, but it also requires a later version of Java. I would recommend 1.4 for better String handling functionality.


My idea was to use replaceAll and preprocess all "::::" to ":: ::" and running trim on each field. This would work as long as trailing and leading blanks would not be used; you can adjust this approach by using a bogus flag character such as "|" and if ("|".equals(field)) field="";

However, not all hope is lost. There are a lot of posted versions of replaceAll type methods on the net, that people were using with earlier versions of the SDK, so you can add that to your code, and you should have a solution.

chantal

Posts: 1
Nickname: chantalack
Registered: Feb, 2003

Re: Problem in Parsing Strings using StringTokenizer Posted: Jun 11, 2003 1:03 AM
Reply to this message Reply
hi,

StringTokenizer treats any string that is given as delimiter argument as a list of one-char-delimiters. So "::" is actually two delimiters, that happen to be equal. Splitting a string at "::" with the StringTokenizer works only if the pattern string+"::"+string+"::" applies regularly and no string is empty. in case a string is zero or one colon is missing you won't get the results you expected.

for good error handling you should:
- either iterate through the characters of the string by yourself and check for double colons, if you want to stick with 1.3 and use no additional package,
- or switch to 1.4 and use the java.util.regex package,
- or use the OROMatcher package from http://www.savarese.org/oro/software/OROMatcher1.1.html,
- or use a parser generator like ANTLR, which will give you even more power if you happen to need more sophisticated parsing.

Chantal

Lee Pederson

Posts: 1
Nickname: lpederson
Registered: Jun, 2003

Re: Problem in Parsing Strings using StringTokenizer Posted: Jun 13, 2003 6:38 PM
Reply to this message Reply

import java.util.*;
public class SuccessiveDelimiterTokenizer {
public static void main(String[] args) {
String delimiter = ":";
int numSucessiveDelimiters = 2;
String text = "999::adm::::mgr::security::flag";
StringTokenizer tok = new StringTokenizer(text, delimiter, true);
while( tok.hasMoreTokens() ) {
String nextToken = tok.nextToken();
if( !nextToken.equals(delimiter) ) {
System.out.println("token = [" + nextToken + "]");
for( int i = 1; (i <= numSucessiveDelimiters)
&& tok.hasMoreTokens(); i++ ) {
nextToken = tok.nextToken();
}
}
else {
System.out.println("Empty token = []");
for( int i = 2; (i <= numSucessiveDelimiters)
&& tok.hasMoreTokens(); i++ ) {
nextToken = tok.nextToken();
}
}
}
}
}

Flat View: This topic has 10 replies on 1 page
Topic: what does static mean? Previous Topic   Next Topic Topic: need someones help

Sponsored Links



Google
  Web Artima.com   

Copyright © 1996-2019 Artima, Inc. All Rights Reserved. - Privacy Policy - Terms of Use