Stats

  • Locked Locked
  • Replies 16
  • Subscribers 143
  • Views 18857
  • Members are here 0
This discussion has been locked.
You can no longer post new replies to this discussion. If you have a question you can start a new discussion

What is the most efficient approach to record each line from a input file ?

Charley Chen

 Hi All,

I use table to record each line & its value , but it will become slower and slower when time left .

Though I can get table[A11100] = list(1000000 2000000 3000000 4000000) very quickly , But must when it finished loop.

What is the best way to do it ?

;write a template file
        ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; 
        getCurrentTime()
        outPort = outfile("test")
        count = 1000000
        fileCount = 1
        for(i 1 count
            fprintf(outPort "1000000 2000000 3000000 4000000\n")
        );   
        close(outPort)
        getCurrentTime()
        ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; 

        ;read each line to record
        ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; 
        nextLine = nil
        inPort = infile("test")
       getCurrentTime()
 when(inPort
      fileCount = 1
      table = makeTable("table" nil)
      while(gets(nextLine inPort)
              qq = parseString(nextLine "\n")
              str = sprintf(nil "A%d" fileCount)
              table[str] = list(1000000 2000000 3000000 4000000)
              fileCount++
      );while   
 );when
      inPort = nil
      getCurrentTime()
        ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

 

Thank you,

Charley

Parents
  • Andrew Beckett

    Charley,

    You don't necessarily need to get it to be the number of lines - it will just minimize the amount of work it has to do.

    Allocating 10 million (or 60 million) of anything is a fairly unusual thing to do. Given that only you know what you're doing here - only you can know how you might be able to determine how many entries you need.

    The number is the number of slots needed. In the case of the strings, you had one string per line in the file (the key into the table). So for a million lines, that's a million strings. For the lists, the number is the number of list cells - and so there's one per list cell per row - hence 4million. gcsummary() tells you how big each type is, and how much total memory has been allocated for each type. As more are needed though, it will automatically allocate more memory - assuming you don't run out.

    If you start to allocate very large numbers, maybe you're running out of memory - it has to store the data somewhere. Particularly if you're running in 32 bit mode, there's a limit of roughly 2^32 bytes (less than 4Gbytes in practice) of available memory - even if your machine has much more. I don't know which version you're using, but in IC5141 only "layout" can be run in 64 bit mode (which would give you access to more memory). From IC614 the "virtuoso" executable can be run in 64 bit mode.

    Regards,

    Andrew.

Reply
  • Andrew Beckett

    Charley,

    You don't necessarily need to get it to be the number of lines - it will just minimize the amount of work it has to do.

    Allocating 10 million (or 60 million) of anything is a fairly unusual thing to do. Given that only you know what you're doing here - only you can know how you might be able to determine how many entries you need.

    The number is the number of slots needed. In the case of the strings, you had one string per line in the file (the key into the table). So for a million lines, that's a million strings. For the lists, the number is the number of list cells - and so there's one per list cell per row - hence 4million. gcsummary() tells you how big each type is, and how much total memory has been allocated for each type. As more are needed though, it will automatically allocate more memory - assuming you don't run out.

    If you start to allocate very large numbers, maybe you're running out of memory - it has to store the data somewhere. Particularly if you're running in 32 bit mode, there's a limit of roughly 2^32 bytes (less than 4Gbytes in practice) of available memory - even if your machine has much more. I don't know which version you're using, but in IC5141 only "layout" can be run in 64 bit mode (which would give you access to more memory). From IC614 the "virtuoso" executable can be run in 64 bit mode.

    Regards,

    Andrew.

Children
No Data

Community Guidelines

The Cadence Design Communities support Cadence users and technologists interacting to exchange ideas, news, technical information, and best practices to solve problems and get the most from Cadence technology. The community is open to everyone, and to provide the most value, we require participants to follow our Community Guidelines that facilitate a quality exchange of ideas and information. By accessing, contributing, using or downloading any materials from the site, you agree to be bound by the full Community Guidelines.