Combine the chilies with 3 tablespoons of the lime juice and 2 tablespoons of the olive oil. Set aside.
Cut the avocados in half. Remove the pits and scoop the flesh into a bowl. Add the remaining 1 tablespoon of lime juice and a big pinch of salt and pepper. Mash until smooth and set aside.
Cut the woody ends off of the asparagus spears. Working with one asparagus spear at a time, place it flat on a work surface and peel the skin off the bottom half of the stalk with a vegetable peeler. Season the peeled asparagus and fish fillets with salt and pepper and brush with the remaining oil.
Heat a large skillet over high heat. Cook the asparagus for 2-4 minutes or until crisp tender. Remove from the skillet and set aside.
Place the fish, skin side down, in the skillet and cook for 4 minutes. Flip and cook for another 1-2 minutes or until cooked through.
Divide the asparagus between 4 plates. Top with the fish fillets, drizzle with the chili oil, and serve with a dollop of avocado cream on the side.