how to remove duplicate subdomains from result of different tools and combine it together?
1. I ran amass and i save output in a txt file sub1.txt
2. I ran subfinder and i save as sub2.txt
3. So i want to combine sub1.txt together with sub2.txt and save as unique.txt
how will i remove duplicate subs(i.e subs appearing in both sub1 and sub2) and append to unique.txt
Hello @Venus, welcome to @webs3c and thank you for your participation!
Regarding your questions:
So i want to combine sub1.txt together with sub2.txt and save as unique.txt
cat sub1.txt > not-unique.txt && cat sub2.txt >> not-unique.txt
how will i remove duplicate subs(i.e subs appearing in both sub1 and sub2
sort not-unique.txt | uniq > unique.txt
Hey @Venus if you want to use just built-in tools in every Linux machine I would do:
cat amass_output.txt subfinder_output.txt | sort -u > final_result.txt
But you can also use
anew TomNomNom’s tool to do this:
cat amass_output.txt subfinder_output.txt | anew -q > final_result.txt