Wow...so many helpful responses...I will try to address them in my reply here...I have many lilies (close to 100) of all kinds and none of the others are showing this kind of damage. There are actually two kinds here I believe, an asiatic and an orienpet, but unfortunately I probably didn't ID mark them when they were planted...I think they are both new lilies to me, planted last year.
I think I will take them out and put them in an isolated pot, let them do their thing and then keep an eye on them for next year, but definitely will remove them today. Thanks for all of the input!