I believe what you have is Ambrosia trifida commonly known as Giant Ragweed. This is not something you want in your garden.

I have a large perennial garden and I know what you mean by not wanting to pull the wrong thing.
I remember pulling up what I thought was wild geranium only to realize after the damage was done that it was the delphinium I had planted the season before. This taught me a valuable lesson . I now mark all the perennials that are a little difficult to recognize when they first come up. Your local garden center will have little plant markers that you can stick in the ground beside your perennial. This has made a big difference in my garden. Hope this helps.