View on GitHub

Bigio

Special .Net collections adapted for huge amount of elements

Download this project as a .zip file Download this project as a tar.gz file

What is Bigio used for?

Bigio is a several collections special adapted to store a lot of elements. Standard List and LinkedList is great choice for the most of cases, but they become useless if you have to work with thousands and millions of elements because of:
1. Ineffective operations because of structure of List and LinkedList
2. Memory overhead of that system collections.

Getting started

There is no nuget-package of Bigio library at this moment, because it still in active development.
If you want to use it you have to clone repository:
Just copy-past that in git client: git clone https://github.com/80LevelElf/Bigio.git
Or download zip file with repository Then open downloaded folder, move to 'bin' folder and copy 'Bigio.dll' file to your project working directory.
Then just specify reference to Bigio.dll from needed projects.

Structure of Bigio

At this moment Bigio consist of this main parts:
1. BigArray
2. BigQueue
3. BigStack
Click to move into separate wiki pages of each collection.

Quck start

All Bigio collections use familiar API and interfaces to make work with it easy.
API of 'BigList' quite similar to system 'List' API, for example:

Insert million elements to random positions:

var list = new BigArray<int>();
Random random = new Random();

for (int i = 0; i < 1000000; i++)
{
    list.Insert(random .Next(list.Count), i);
}

After that we can find index of some element:

list.IndexOf(random.Next(list.Count));

Remove last entry of some element:

list.RemoveLast(random.Next(list.Count));

Or find first event element with LINQ(or default(T) if all elements are odd):

var firstEven = list.FirstOrDefault(i => i % 2 == 0);

Is there any alternative solutions?

Of course they are! The most famous alternative is Wintellect's Power Collections of Wintellect company. It's great solution developed by clever people, but in most of cases Bigio will be faster:
All search estimations measured for collections with 1.000.000 elements.
We compare Bigio.BigArray<T> and PowerCollections.BigList<T>

Estimation much be different in different computers(depend on processor and count of cores inside it), compilers and the phases of the moon.

Search:

IndexOf(item):

Iterations Bigio Wintellect Microsoft List
10 1 1 0
100 0 0 8
1000 1 28 0
10000 57 2503 62

LastIndexOf(item):

Iterations Bigio Wintellect Microsoft List
1 4 229 2
2 2 438 1
5 8 1117 6
10 16 2883 15

Find(predicate):

Iterations Bigio Wintellect Microsoft List
10 0 0 0
100 0 0 0
1000 3 39 6
10000 398 3126 295

FindLast(predicate):

Iterations Bigio Wintellect Microsoft List
1 20 275 4
2 12 790 7
5 35 1550 17
10 142 2669 44

FindAll(predicate):

Iterations Bigio Wintellect Microsoft List
2 43 228 9
5 140 1480 75
7 309 2669 158
Adding and insertions:

Add(item):

Iterations Bigio Wintellect Microsoft List
10000 0 0 0
100000 13 4 0
1000000 100 59 5
10000000 1162 629 58
100000000 12468 6087 588

Insert(item) in the start position:

Iterations Bigio Wintellect Microsoft List
100 1 0 0
1000 0 0 0
10000 9 1 17
100000 60 17 1579
1000000 642 203 until the end of time
10000000 7805 2422 until the end of time

Insert(item) in the middle position:

Iterations Bigio Wintellect Microsoft List
100 1 1 0
1000 0 8 0
10000 12 130 3
100000 83 15385 489

Insert(item) in the random position:

Iterations Bigio Wintellect Microsoft List
100 1 0 0
1000 0 0 0
10000 12 8 7
100000 110 135 1471
1000000 2095 3001 If you wait for the end - tell me the result

AddRange(range):

Iterations Bigio Wintellect Microsoft List
1000 0 1 0
10000 19 17 14
100000 154 254 127
1000000 1503 2239 1138

InsertRange(range) in random position:

Iterations Bigio Wintellect Microsoft List
100 0 0 0
1000 1 3 1
10000 37 78 107
100000 489 673 We'll never know
Other:

BinarySearch(item, 0, collectionLength)

Iterations Bigio Wintellect Microsoft List
100 0 4 0
1000 6 8 1
10000 73 107 5
100000 908 1197 71

Foreach through full collection

Iterations Bigio Wintellect Microsoft List
1 63 108 25
5 810 1508 276
10 2701 4593 1141

For through full collection

Iterations Bigio Wintellect Microsoft List
1 1046 789 23
4 12638 7838 232

Reverse() all elements

Iterations Bigio Wintellect Microsoft List
1 1 1299 0
5 24 - 19
10 69 - 46
15 150 - 94

Memory estimation (memory size of process with only one collection):

Count of elements Bigio Wintellect Microsoft List
10000000 59.8 Mb 73.3 Mb 106.3 Mb
100000000 620 Mb 689.5 Mb 641 Mb
1000000000 6 241 Mb 6 955 Mb Out of memory

As you can see Bigio give you much faster and smaller solution for big collection than PowerCollections.