In this BoardGameGeek thread here another player had the same questions. The only problem is that they are playing the full game, and there are things that do not apply to you (moreover there will be spoilers that maybe you want to avoid).
So I'll try to sum up his rules that apply to Game 4:
To make the game shorter, I would say start on location 2, and reduce the number of villains. He recommends from experience to divide the Villains in three categories, from easy to hard. I report his list, with only the villains you discovered:
Easy Villains:
- Crabble
- Quirrel
- Death Eater
Medium Villains:
- Draco Malfoy
- Basilisk
- Dementor
- Tom Riddle
Hard Villains:
- Lucius Malfoy
- Peter Pettigrew
I personally disagree with the dementor being a medium villain.
But regardless:
Remove one villain from each category and you'll have shorter game.
Also start from location 2 to make it a little harder.
As for player hands, I would start with 4 cards added from the store each. There are different ways you can do this:
- Pull out 8 cards from the store and choose who gets which (try to still do 4 each)
- Assign yourelf 20 influence points and choose the cards (I would ban Petrificus Totalus though, as it's quite strong (Maybe not in 2 players though))
- You could also stratify the cards (like 1 card expensive 5+, 1 card 3-4, and 1 card 2 for each players) but that takes a longer setup so maybe you won't like it
To conclude:
- -3 Villains
- -1 Location
- +6 Random cards
There are other different tweaks that the post gives, but I haven't included them as they could be spoilers.
EDIT: Also, when you reach further games let me know so I can update this summary with the new information.
Credit goes to BGG User ranissi for these rules.