As a data analyst or data scientist, you’re constantly working with data, and often that data is stored in lists. While you probably know the basics, some powerful, lesser-known operations can save you time and make your code more efficient.
Over my two years working with Python, I’ve compiled a list of handy operations that have been incredibly useful for various tasks. I’m excited to share them with you!
1. Shuffle a List
Need to randomize the order of items in your list? Don’t try to build a complex loop; just use the random module. This is super handy for tasks like creating randomized test data or shuffling a deck of cards in a program.
Python
import random
my_list = ['start', 1, 2, 3, 4, 5, 6, 7, 8, 9, 'end']
random.shuffle(my_list)
print(my_list)
Output: [2, 3, 'start', 1, 4, 9, 8, 6, 'end', 7, 5] (output will vary)
For example, when creating a toy dataset for an e-commerce site, I needed to randomize user actions like “Search,” “Click,” and “Add to Cart” to make the data look more realistic. Shuffling the list of actions was the perfect solution!
2. Reverse a List
Reversing a list is simpler than you might think. Python’s slice notation offers a clever and concise way to do it without any extra functions.
Python
my_list = ['start', 1, 2, 3, 4, 5, 6, 7, 8, 9, 'end']
reversed_list = my_list[::-1]
print(reversed_list)
Output: ['end', 9, 8, 7, 6, 5, 4, 3, 2, 1, 'start']
This trick creates a new reversed list without modifying the original, which is a great way to maintain data integrity.
3. Generate All Combinations from a List
For those times when you need to find every possible unique combination of elements, the itertools module is your best friend. This is useful for tasks like generating all possible pairs from a list of people or creating all subset scenarios for an analysis.
⚠️ Warning: Be cautious with this one! It can become very slow on large lists, as the number of combinations grows exponentially.
Python
import itertools
stuff = [1, 2, 3]
for L in range(0, len(stuff) + 1):
for subset in itertools.combinations(stuff, L):
print(subset)
Output:
()
(1,)
(2,)
(3,)
(1, 2)
(1, 3)
(2, 3)
(1, 2, 3)
4. Get a Random Sample from a List
Sometimes you don’t need to shuffle the whole list; you just need to grab a few random items from it. The random.sample() function is perfect for this.
Python
from random import sample
my_list = [1, 2, 3, 4, 5]
random_sample = sample(my_list, 3)
print(random_sample)
Output: [3, 2, 4] (output will vary)
This is ideal for selecting a random subset for a small test run or for creating a quick demo.
5. Find the Difference Between Two Lists
Want to find out which elements are in one list but not the other? The most efficient way to do this is by converting your lists to sets. Sets are designed for fast comparisons and unique element storage.
Python
cols = ['start', 'finish', 'version', 'country', 'device']
test = ['start', 'finish']
difference = list(set(cols).difference(test))
print(difference)
Output: ['device', 'country', 'version'] (order may vary)
This is a powerful method for comparing two lists of IDs, columns, or any other data to quickly identify discrepancies.
6. Create a List of Random Numbers
Generating a list of random numbers is a common task in data science. Using NumPy is the most common and efficient way to do it. It’s perfect for creating synthetic data or running simulations.
Python
import numpy as np
random_numbers = list(np.random.rand(10, 1).ravel() * 1000)
random_numbers = [int(x) for x in random_numbers]
print(random_numbers)
Output: [639, 170, 381, 44, 448, 178, 145, 17, 514, 607] (output will vary)
7. Sort a List
This is a fundamental operation, but it’s worth including because of its flexibility. The .sort() method sorts the list in place, which means it modifies the original list.
Python
my_list = [639, 170, 381, 44, 448, 178, 145, 17, 514, 607]
my_list.sort()
print(my_list)
Output: [17, 44, 145, 170, 178, 381, 448, 514, 607, 639]
If you want to sort in reverse order, just use the reverse parameter:
Python
my_list.sort(reverse=True)
print(my_list)
Output: [639, 607, 514, 448, 381, 178, 170, 145, 44, 17]
8. Find the Position of the First Occurrence
When you need to find the index of the first element in a list that meets a certain condition, NumPy’s np.argmax() function is a lifesaver. It returns the index of the first occurrence of the maximum value, which you can leverage to find the first element that’s True after a conditional check.
Python
import numpy as np
my_list = [17, 44, 145, 170, 178, 381, 448, 514, 607, 639]
index = np.argmax(np.array(my_list) >= 400)
print(index)
Output: 6
This operation is particularly helpful for data visualization. For instance, I used it to find the exact point on a logistic regression curve where the probability crossed a certain threshold, allowing me to draw a vertical line on the plot.
I hope you enjoyed these tips and found them as useful as I have. If you want to learn more about data science and analytics, be sure to follow along for future posts!
What’s your favorite list operation that you use every day? Let me know in the comments!
